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ABSTRACT 

This  thesis  surveys  the  Qualified  Products  Lists  (QPL) 
program.  It  provides  an  introduction  to  standardization  and 
specification  by  giving  their  definitions  and  explaining 
their  role  in  the  government  acquisition  process.  The 
Defense  Standardization  and  Specification  Program  is  briefly 
outlined  to  provide  a  framework  of  understanding  of  the 
context  in  which  the  QPL  process  works.  The  intended 
application  of  the  prequalif ication  process  is  covered  along 
with  a  history  of  the  QPL  program.  Summaries  of  two  prior 
studies  on  the  effectiveness  of  the  QPL  program  are 
discussed.  Included  also  is  testimony  from  the  Hearings 
before  the  Senate  Committee  on  small  business  in  1984 
concerning  competition  for  parts  procurement. 

Interview  comments  obtained  from  vendors  and  contractors 
dealing  in  the  electronic  business  sector  are  compiled  with 
their  suggestions  pertaining  to  the  QPL  process.  The  last 
chapter  contains  analyses  of  the  previous  studies  contrasted 
to  research  conducted,  recommendations  for  program 
improvement  and  conclusions. 
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I.   INTRODUCTION  TO  QUALIFIED  PRODUCTS  LISTS 

A.   PURPOSE  OF  PAPER 

There  are  many  instructions,  policies,  regulations,  and 
directives  governing  the  federal  acquisition  process.  A 
myriad  of  studies,  research  papers,  theses,  committee 
reports,  and  independent  analyses  have  been  conducted 
concerning  the  effectiveness  of  various  aspects  of  this 
process.  However,  the  Qualified  Products  List  (QPL) 
program,  currently  under  the  cognizance  of  the  Defense 
Standardization  and  Specification  Program,  has  received 
relatively  little  research  attention.  Each  federal  agency 
maintains  administrative  control  over  its  own  QPLs.  Until 
recently,  there  was  no  central  administrative  control  over 
the  entire  federal  program.  Even  now,  there  are  no 
effective  or  visible  management  statistics  to  measure 
program  benefits  or  determine  a  cost-effective  analysis. 

Investigation  has  revealed  that  only  two  studies  have 
been  conducted  by  the  Department  of  Defense  on  the  QPL 
program.  The  DOD  is  by  far  the  largest  user  of  QPLs.  In 
each  study  problems  were  noted  and  recommendations  made. 
However,  no  indication  of  follow-up  was  discovered.  The  QPL 
program  has  been,  and  continues  to  be,  controversial.  Small 
businesses  claim  the  program  is  unfair  by  limiting 
competition  and  have  brought  their  complaints  to  the  Small 


Business  Administration  and  to  Congress.  Because  of  the 
attention  the  QPL  program  has  received  by  Congress  and  the 
SBA,  QPL  directives  have  been  updated  and  resources  budgeted 
to  better  administer  the  program.  But  is  the  program 
effective  in  accomplishing  its  objectives? 

In  the  last  two  years  another  area  of  QPL  application 
has  received  much  attention,  the  electronic  sector  of 
semiconductors.  The  industry  is  outpacing  QPL  procurement 
procedures.  With  the  growing  importance  of  integrated 
circuits  in  major  defense  systems,  this  nation  should  adopt 
business  procedures  that  allow  it  to  obtain  the  highest 
quality,  most  advanced  electronic  products.  How  can  the  QPL 
process  achieve  these  objectives?  This  paper  will  explore 
these  questions  and  provide  some  suggestions. 
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II.   STANDARDS  AND  SPECIFICATIONS  IN  THE  GOVERNMENT 

ACQUISITION  PROCESS 


A.   INTRODUCTION 

This  chapter  will  provide  an  introduction  to  the  purpose 
and  importance  of  standards  and  specifications  in  the 
government  acquisition  process  and  give  an  understanding  for 
the  purpose  of  Qualified  Products  Lists  (QPLs) .  Definitions 
of  standards  and  specifications  are  given  along  with  a  brief 
history  of  their  implementation  in  the  government  acquisi- 
tion process.  The  Defense  Standardization  and  Specification 
Program  (DSSP)  is  also  briefly  outlined.  QPLs  will  be 
introduced  at  the  end  of  the  chapter. 

The  acquisition  process  is  the  initial  phase  of  the 
logistic  support  cycle  in  the  Federal  supply  support  estab- 
lishment. This  process  has  to  be  responsive  to  a  myriad  of 
requirements  ranging  from  simple  general  hardware  items  such 
as  nuts  and  bolts  to  complex  and  ambiguous  research  and 
development  of  human  resource  projects.  For  the  acquisition 
process  to  fulfill  its  role  in  the  logistic  support  cycle, 
requirements  must  be  defined  in  such  a  way  that  a  conceptu- 
alized need  can  be  conveyed  to  potential  buyers  through  a 
competitive  process.  An  exact  understanding  of  this 
requirement  must  be  sufficiently  conveyed  so  that  a  contrac- 
tor can  prepare  estimates  that  can  be  fairly  and  equitably 
evaluated  prior  to  an  award  of  a  contract.    [Ref.  l:p.  2] 


Standards  and  specifications  are  the  current  means  used  by 
the  Federal  Government,  and  more  specifically  the  Department 
of  Defense  (DOD) ,  to  define  these  requirements.  The  DOD 
Index  of  Specifications  and  Standards  (DoDISS)  currently 
lists  more  than  45,000  active  standardization  documents 
prepared  by  DOD  activities,  other  federal  agencies,  or 
industry  groups.   [REf.  2:p.  10] 

B.   DEFINITIONS 

1.   Specifications 

Specifications  are  documents  prepared  specifically 
to  support  acquisition  and  cover  items  which  vary  in 
complexity  from  uniform  chevrons  to  missile  weapon  systems. 
They  establish  requirements  in  terms  of  complete  design 
details  or  in  terms  of  performance,  but  in  most  cases  in 
terms  of  both  design  and  performance.  Design  specifications 
attempt  to  define  the  end  item  in  terms  of  its  physical 
characteristics.  Generally  they  state  precise  measurements, 
tolerance,  materials,  production  process,  and  finished 
product  tests,  quality  control  and  inspection  requirements. 
Performance-type  specifications  take  on  the  form  of  a 
performance  characteristic  or  a  functional  description.  It 
expresses  requirements  in  terms  of  functions  to  be  performed 
such  as  degrees  of  precision,  speed  of  operation,  mainte- 
nance levels  expected  (e.g.,  mean  time  between  failures), 
general  tests  and  quality  standards.  Performance-type 
specifications  are  normally  associated  with  more  complex, 
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technologically  advanced  requirements.  [Ref.  l:p.  3] 
Specifications  should  establish  requirements  insofar  as  is 
practicable  in  terms  of  performance  so  that:  1)  they  do  not 
restrict  creativity  in  meeting  specifications,  2)  they 
permit  solicitations  of  competitive  bids  from  the  largest 
segment  of  industry,  and  3)  they  place  greater  responsibili- 
ty on  the  contractor  to  achieve  the  performance  required. 
Specifications  may  cover  a  single  item  such  as  a  camera  or 
millions  of  items  such  as  bullets.  To  fulfill  their 
purpose,  specifications  should  be  tailored  for  each 
application. 

2 .   Standards 

Standards  are  documents  that  establish  engineering 
and  technical  requirements  for  processes,  procedures, 
practices  and  methods  that  have  been  adopted  as  standard. 
Their  purpose  is  to  control  variety.  They  may  cover 
materials,  features  of  items,  engineering  practices,  defini- 
tions, nomenclature,  test,  inspection,  packaging  and  preser- 
vation methods.  Standards  represent  the  best  solution  or 
preferred  solution  to  recurring  design,  engineering  and 
other  logistic  problems.  Standards  function  in  acquisition 
through  specifications.  They  disclose  or  describe  the 
technical  feature  of  an  item  in  terms  of  what  it  is  and  what 
it  will  do.  In  contrast,  the  specification  for  the  same 
item  describes  it  in  terms  of  the  more  complex  description 
of  requirements  for  acquisition.   Standards  are  referenced 
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in  specifications  for  those  design  requirements  which  are 
essential  to  achieve  the  design  objectives  (i.e.,  inter- 
changeability,  compatibility,  reliability,  and  maintaina- 
bility) .  [Ref.  3:p.  3-2]  Simply  stated,  standards  specify 
item  form,  its  physical  shape,  item  fit,  description  of 
item's  input/ output  characteristics,  and  item  function. 

As  an  example  of  the  relationship  between  specifica- 
tions and  standards,  a  specification  for  spark  plugs  would 
reference  the  screw-thread  standard  and  reach  standard 
(operating  temperature  range) ,  to  insure  optimal  engine 
performance  and  interchangeability  of  spark  plugs  produced 
by  different  manufacturers.   [Ref.  2:p.  5] 

C .   BACKGROUND 

Standardization  in  the  military  departments  has  been  an 
evolutionary  process.  World  War  I  pointed  out  the  need  for 
standardization.  The  U.S.  Armed  Forces  were  repeatedly 
frustrated  because  of  the  differences  in  such  matters  as 
operating  procedures,  tactics,  flight  safety  and  aircraft 
design.  In  1919  the  Departments  of  War  and  Navy  established 
a  Joint  Aeronautical  Board  to  address  these  issues.  This 
board  could  be  considered  a  forerunner  to  a  military  speci- 
fication and  standardization  program.  [Ref.  4:p.  62]  The 
Aeronautical  Board  was  replaced  by  the  Aeronautical 
Standards  Group  which  in  1941  began  publishing  a  Joint  Army- 
Navy  series  of  specifications  and  standards.  In  1942  the 
Joint  Army-Navy  Specification  Board  was  formed  to  provide 
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cooperation  between  the  services  in  both  procurement  and 
standardization.  This  Board  established  procedures  and 
format  controls  over  the  Joint  Army-Navy  series  of 
specifications . 

In  February  1953  Public  Law  436,  the  Cataloging  and 
Standardization  Act  (codified  title  10  U.S.C.  Chapter  145- 
Section  2451-6),  was  translated  into  DOD  Directive  4120.3. 
The  Defense  Standardization  Program  was  established.  The 
defense  Supply  Management  Agency  was  then  assigned  manage- 
ment control.  It  established  the  military  series  of  speci- 
fications and  standards  to  replace  the  Joint  Army-Navy 
series  and  all  other  series  in  use  by  the  military  depart- 
ments. At  this  time,  the  use  of  military  specifications  was 
made  mandatory  for  all  acquisitions. 

The  basic  DOD  Direction  412  0.3  has  been  revised  many 
times  to  effect  improved  standardization  management  within 
the  military  series.  During  the  1960's  revisions  provided 
for  coordination  of  the  Defense  Standardization  Program  with 
the  General  Services  Administration,  the  Aeronautical 
Standards  Group,  and  other  government  agencies  and  bureaus. 
A  1965  revision  also  provided  for  industry  participation  to 
avoid  duplication  of  effort  and  to  obtain  the  benefit  of 
technological  and  managerial  skill.  Current  DOD  standardi- 
zation and  specification  policies  and  operating  procedures 
are  published  in  the  Defense  Standardization  Manual  4120. 3M. 
It  is  the  result  of  many  years  of  development.   Today  the 
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Defense  Standardization  Program  is  the  most  diversified  and 
largest  standardization  activity  in  the  world.  [Ref.  4:pp. 
62-63] 

Why  has  so  much  effort  been  made  to  more  effectively 
manage  standardization  in  government  acquisitions  and  to 
increase  its  usage? 

D.   BENEFITS  OF  STANDARDIZATION 

The  primary  purposes  for  applying  standardization 
principles  follow. 

1.  Standardization  conserves  resources.  It  reduces  the 
unnecessary  and  inefficient  proliferation  of  generally 
similar  types,  kinds,  sizes,  and  styles  of  items,  thus 
avoiding  costs  of  specifying  new  items.  Larger  dis- 
counts can  be  realized  by  making  larger  purchases. 
Better  prices  may  also  be  realized  by  the  availability 
of  additional  competitive  sources. 

2.  Standardization  is  a  base  upon  which  to  certify. 
Standardization  of  parts,  components  and  subassemblies 
reduces  the  risks  associated  with  developing  and  pro- 
ducing new  products  and  services.  Standardized 
products  have  a  historical  record  of  usefulness, 
reliability  and  performance. 

3.  Standardization  can  provide  a  "stepping  stone"  for 
evolutionary  improvements.  It  can  promote  technologi- 
cal growth  by  providing  an  accepted,  reliable  founda- 
tion for  improvements  and  innovations. 

4.  Standardization  simplifies.  It  conserves  resources  by 
minimizing  and  simplifying  training,  technical  data, 
engineering  and  support  requirements.  Standard  items 
significantly  reduce  expenditure  of  research,  develop- 
ment, test  and  evaluation,  and  logistics  support 
resources.  The  number  of  purchase  orders  and  receiv- 
ing inspections  can  be  reduced  as  well  as  inventory 
and  inventory  carrying  costs.   [Ref.  2:p.  3] 

Also  intangible  benefits  are  achieved,  which  include: 

1.  Standardization  educates.  They  set  forth  quality 
goals  or  ideals,  for  the  guidance  of  manufacturers  and 
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users  alike.   They  are  invaluable  to  the  manufacturer 
who  wishes  to  enter  a  new  field. 

2.  Standardization  provides  a  common  language  between 
buyers  and  sellers.  They  improve  quality  control 
based  on  accepted  and  explicit  specifications.  [Ref. 
5:p.  ix] 

Now  that  standards  and  specifications  have  been  defined 

and  their  purposes  and  benefits  covered,  an  overview  of  the 

Defense  Standardization  and  Specification  Program   (DSSP) 

will  be  given.   An  introduction  to  the  Qualified  Products 

Lists  (QPLs)  follows. 

E.   DSSP 

The  DSSP  is  a  decentralized  program  with  overall  DOD 
policy,  guidance,  and  administration  centered  in  the  Office 
of  the  Assistant  Secretary  of  Defense  for  Acquisition  and 
Logistics  (OASDA&L) .  Overall  management  of  standardization 
policies,  procedures,  and  guidance  is  the  responsibility  of 
the  Director,  Standardization  and  Acquisition  Support, 
within  OASDA&L.  Daily  operations  are  delegated  to  the 
Director,  Defense  Materiel  Specifications  and  Standards 
Office  (DMSSO) .  Within  each  service  and  the  Defense 
Logistics  Agency  (DLA) ,  a  Departmental  Standardization 
Office  (DEPSO)  has  been  established  to  manage  those  portions 
of  the  DSSP  assigned  to  the  respective  Department  and 
Agency. 

Products  used  by  the  military  are  grouped  into  logical 
families,  such  as  space  vehicle  components,  flight 
instruments  and  land  mines,  and  are  identified  as  Federal 
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Supply  Classes  (FSCs) .  Management  and  engineering  prac- 
tices, such  as  reliability,  safety,  and  configuration 
management,  are  identified  as  Standardization  Areas.  For 
each  FSC  and  Standardization  Area,  a  military  organization 
known  as  an  Assignee  Activity  (for  FSCs)  and  Lead  Service 
Activity  (for  Areas)  is  delegated  the  responsibility  for 
analyzing,  planning  for,  and  insuring  that  optimal  standar- 
dization is  accomplished. 

Development  of  the  actual  specifications,  standards,  and 
related  documents  is  performed  by  DOD  organizations  known  as 
Preparing  Activities.  It  is  the  Preparing  Activity's 
responsibility  to  develop,  maintain,  and  coordinate  individ- 
ual DSSP  documents,  and  to  insure  that  they  meet  mission 
requirements.  [Ref.  2: pp.  10-11]  Procedures  for  preparing 
and  coordinating  the  documents  are  outlined  in  DOD  4120. 3M. 
Also,  an  organizational  chart  showing  the  relationship  and 
responsibilities  of  standardization  management  is  shown  on 
page  4-24  DOD  4120. 3M. 

F.   THE  CONCEPT  OF  QPLS 

Standards  and  specifications  are  an  excellent  means  for 
conveying  requirements  to  prospective  sellers  and  buyers. 
However,  during  the  1920 's  as  specifications  became  more 
complex,  due  to  advances  in  technology  and  greater  complex- 
ity in  weapon  systems,  the  time  between  creation  of  an  item 
specification  and  delivery  of  completed  item  became  greater 
and  greater.    Therefore,   some  time-critical  requirements 
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experienced  delays  in  being  met.  It  was  noted,  first  by  the 
Navy,  that  the  delays  were  basically  caused  by  the  testing 
requirements  to  insure  that  the  item  conformed  to  the  speci- 
fications. A  way  was  needed  to  make  the  procurement  system 
more  responsive  to  critical  requirements.  Out  of  that  need 
was  created  the  "Navy  Qualified  Products  Lists."  Its 
conceptualized  purpose  was  to  insure  the  procuring  agency  of 
timely  delivery  of  products  with  reasonable  assurance  that 
they  would  be  satisfactory  for  their  intended  use. 

The  process  was  designed  to  work  in  the  following 
manner.  The  government  agency  (i.e.,  Navy)  would  first  have 
to  have  an  item  requirement.  That  requirement  had  to  be 
translated  into  a  specification.  The  Navy  would  then 
announce  to  prospective  contractors  the  need  for  qualified 
manufacturers  of  that  item.  The  manufacturers  would  supply 
their  own  resources  to  ,  build  the  item  and  test  it  under 
government  supervision,  with  no  guarantee  of  subsequent  pro- 
curements of  that  item.  Upon  satisfactory  completion  of 
testing,  that  manufacturer  would  become  "qualified"  to 
produce  that  particular  item.  Then,  whenever  the  government 
required  that  particular  item,  only  the  qualified  manufac- 
turers would  be  considered  in  the  bidding  for  the  contract. 
Because  these  qualified  manufacturers  had  already  proven 
themselves  capable  of  producing  the  item,  only  minimal 
quality  assurance  inspections  were  considered  necessary  for 
subsequent  procurements. 
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As  the  other  military  departments  began  qualifying  some 
of  their  procurements,  conditions  for  qualification  testing 
were  established  by  the  DOD.  The  directive  stated  that  a 
qualification  requirement  may  be  included  in  a  specification 
when  one  or  more  of  the  following  conditions  existed: 

1.  The  time  required  for  testing  in  connection  with  pro- 
duction would  unduly  delay  delivery  of  the  supplies 
being  purchased; 

2.  The  tests  would  require  special  equipment  not  commonly 
available; 

3.  The  costs  of  repetitive  testing  would  be  excessive;  or 

4.  The  interest  of  the  Government  requires  assurance, 
prior  to  award,  that  the  product  is  satisfactory  for 
its  intended  use.   [Ref.  6:p.  4] 

The  QPL  program  was  designed  to  save  time  in  the  procurement 

of  products  and  provide  some  assurance  of  proper  operation 

in  use. 

G.   SUMMARY 

This  chapter  has  covered  standards  and  specifications  to 
lay  the  foundation  for  an  understanding  of  the  government 
acquisition  process  and  the  premises  for  the  creation  of  the 
QPL  program.  This  foundation  is  considered  necessary  to 
fully  understand  and  appreciate  the  problems  and  recommenda- 
tions that  will  be  brought  up  throughout  the  course  of  this 
paper. 

Today  the  qualification  program  is  a  viable  alternative 
in  the  acquisition  of  products.  The  program  has  continuous- 
ly changed  over  the  years,  as  problems  have  surfaced  and 


18 


recommendations  have  been  made.  There  are  now  approximately 
12  00  QPLs  families  with  the  most  active  sector  being  elec- 
tronics. As  with  all  programs,  QPLs  are  only  as  effective 
as  their  application.  Observations  on  the  qualification 
program  and  its  current  application  in  the  high-tech  elec- 
tronics sector  will  be  covered  in  Chapter  IV.  But  first, 
the  history  of  QPLs  will  be  discussed  along  with  various  DOD 
reports  done  on  the  past  effectiveness  of  the  program. 
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III.   HISTORY  OF  THE  OPL 

A.   INTRODUCTION 

This  chapter  will  provide  a  brief  history  of  Qualified 
Products  Lists  (QPLs) .  Current  Department  of  Defense  direc- 
tives will  be  cited  which  govern  the  policies  and  procedures 
of  QPLs.  The  definition  of  QPLs,  its  purpose  and  intent 
will  be  stated.  Two  Department  of  Defense  (DOD)  studies 
have  been  conducted  concerning  the  effectiveness  of  the  QPL 
program.  First  a  1968  study  will  be  discussed,  and  its  main 
findings  and  recommendations  for  improvement  of  the  program 
will  be  summarized.  In  1979  a  second  study  of  the  QPL 
program  was  conducted  by  the  Defense  Materiel  Specifications 
and  Standards  Office.  That  study's  findings  and  recommenda- 
tions will  also  be  summarized.  As  brought  out  in  these 
studies,  a  major  criticism  of  the  QPL  program  was  that  QPLs 
restricted  competition  for  government  procurements.  That 
debate  peaked  during  Senate  hearings  before  the  Committee  on 
Small  Business  in  April  of  1984.  Specific  arguments  brought 
up  at  the  hearings,  both  for  and  against  QPLs,  will  also  be 
covered.  Recommendations  mentioned  at  the  hearings  for 
improvement  of  the  program  will  also  be  summarized.  A 
summary  and  conclusions  drawn  from  the  studies  and  hearings 
will  complete  the  chapter. 
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B.  BACKGROUND 

In  the  late  1920'  s  a  decision  rendered  by  the 
Comptroller  General  of  the  United  States,  permitted  the  Navy 
to  establish  a  list  of  approved  materials  for  use  in 
procurement  of  military  items.  Otherwise,  delivery  could  be 
delayed  while  testing  was  conducted  to  ensure  conformance 
with  performance  specifications.  [Ref.  6:p.  2]  This  list 
was  called  the  "Navy  Acceptable  List  of  Approved  Materials" 
and  was  later  changed  to  the  "Navy  Qualified  Products  List." 
Following  World  War  II,  the  Army,  Navy  and  Air  Force  estab- 
lished a  joint  list  known  as  the  "Military  Qualification 
Products  List,"  to  be  used  in  connection  with  military  pro- 
curements. The  policies  and  procedures  applied  to  the 
Military  and  Federal  Qualified  Products  Lists  are  based  on 
the  same  Comptroller  General  decisions  as  the  original  Navy 
Acceptable  List.  The  present  statutory  authority  for 
Qualified  Products  Lists  is  found  in  the  U.S.  code  2452. 

C.  QUALIFICATION  DIRECTIVES 

Pursuant  to  this  Act,  authority  to  establish,  publish, 
review  and  revise  Qualified  Products  Lists  was  delegated  to 
the  military  departments.  Specific  provisions  for  the 
conditions  under  which  QPLs  may  be  issued  were  included  in 
the  Armed  Services  Procurement  Regulations  (ASPR) ,  now 
titled  Federal  Acquisition  Regulations  (FAR) .  General 
information,  procedures,  format  and  provisions  governing  the 
qualification    process    were    delineated    in    Defense 

21 


Standardization  Manual  M200,  now  titled  Defense  Standardiza- 
tion Manual.  DOD  4120. 3-M,  "Defense  Standardization  and 
Specification  Program."  Amplifying  information  on  the  QPL 
program  is  found  in  Provisions  Governing  Qualification 
(Qualified  Products  Lists)  SD-6,  "Defense  Standardization 
and  Specification  Program." 

D.  DEFINITION  OF  QUALIFICATION 

Qualification  is  defined  as,  "the  entire  process  by 
which  products  of  manufacturers  or  distributors  are  examined 
and  tested  in  accordance  with  requirements  specified  in  a 
federal  or  military  specification  and  then  identified  on  a 
Qualified  Products  Lists."  [Ref.  7:p.  1]  A  QPL  is  then  a 
list  of  products  designated  in  their  specifications  that 
must  be  manufactured  by  a  qualified  company  prior  to  govern- 
ment procurement.  Each  qualified  product  or  part  lists  the 
names  and  addresses  of  manufacturers  qualified  to  produce 
it. 

E.  PURPOSE  OF  QUALIFICATION 

The  specification  is  the  only  medium  for  establishing  a 
requirement  for  qualification.  Since  most  specifications 
are  based  on  performance  requirements,  the  possible  varia- 
tions in  design,  quality  and  the  nature  of  products  are  such 
that  it  is  deemed  impractical  to  procure  them  solely  on 
conformance  tests  without  unduly  delaying  delivery.  For 
some  items,  simple  non-destructive  conformance  tests  do  not 
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exist.  The  purpose  of  qualification  therefore,  is  to 
provide  for  long,  complex  or  expensive  evaluations  and  tests 
prior  to,  and  independent  of,  any  acquisition,  and  thus 
eliminate  delivery  delay.  Qualification  is  also  intended  to 
reduce  unit  product  costs  and  to  improve  readiness  through 
assured  continuous  availability  of  designated  products.  To 
ensure  quality,  reliability,  and  safety  of  specific  products 
or  families  of  products,  destructive  qualification  tests  are 
required  prior  to  the  opening  of  bids  or  the  award  of  nego- 
tiated contracts.  [Ref.  7:p.  1]  Testing  of  a  product  for 
compliance  with  the  requirements  of  a  specification  in 
advance  of,  and  independent  of  any  specific  procurement 
action,  is  identified  as  qualification  testing  in  the  speci- 
fication. Products  that  successfully  pass  the  required 
tests  are  included  on  a  QPL  appropriately  identified  and 
related  to  the  pertinent  specification.  Preparing  activi- 
ties identified  in  the  specification  are  responsible  for 
qualification.  To  establish  a  QPL  an  approved  and  dated 
military  or  federal  specification  or  non-government  standard 
must  exist  which  requires  qualification  and  sets  forth  the 
qualification  examination,  tests  and  criteria  for  retention. 

F.   REQUIREMENTS  FOR  QUALIFICATION 

Approval  for  justification  of  qualification  is  granted 
by  the  departmental  standardization  office  and  the  Defense 
Product  Standards  Office  (DPSO)  ,  when  one  or  more  of  the 
following  conditions  exist: 
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1.  The  time  required  to  conduct  one  or  more  of  the  exam- 
inations and  tests  to  determine  compliance  with  all 
the  technical  requirements  of  the  specification  will 
exceed  3  0  days  and  would  unduly  delay  delivery  of  the 
products  being  purchased.  Use  of  this  justification 
should  advance  product  acceptance  by  at  least  3  0  days. 

2.  Quality  conformance  inspection  would  require  special 
equipment  not  commonly  available. 

3.  Qualification  covers  life  survival  or  emergency  life- 
saving  equipment  (see  FAR  9.304b). 

4.  The  application  is  critical;  failure  of  the  part  or 
equipment  would  jeopardize  successful  completion  of 
the  mission  or  pose  a  significant  risk  to  life  or 
property.   [Ref.  3:p.  4-3] 

Prior  to  inclusion  of  qualification  in  a  specification, 

the  preparing  activity  shall  determine  that: 

1.  There  is  no  other  pracricable  way  of  obtaining  conclu- 
sive evidence  of  the  availability  of  products  meeting 
the  requirements  of  the  specification  in  a  reasonable 
time,  prior  to,  and  independent  of,  acquisition. 

2.  Two  or  more  sources  are  available  and  willing  to  sub- 
mit their  products  for  qualification. 

3.  Test  facilities  and  resources  are  available  to  estab- 
list  and  maintain  the  QPL  adequately  and  without 
delay. 

4.  The  estimated  cost  of  testing  and  evaluation  has  been 
developed.   [Ref.  3:p.  4-4] 

When  qualification  is  determined  to  be  required,  it 
shall  be  included  as  a  specification  requirement  at  the  time 
of  initial  document  promulgation. 

When  instituted,  the  qualification  process  was  intended 
for  use  on  a  selected  basis  and  only  when  the  requirement 
could  be  justified.  However,  by  1965  the  qualification 
process  was  applied  in  approximately  1,561  general  and  sub- 
sidiary  specifications,   5%  of  the  existing   federal   and 
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military  specifications.  And  the  percentage  of  specifica- 
tions requiring  prequalif ication  was  increasing.  This 
growing  number  of  QPLs,  coupled  with  criticism  of  the 
program  by  government  contractors,  indicated  possible  misun- 
derstanding by  military  departments  of  the  role  of  qualifi- 
cation in  government  procurements.  This  possible 
misunderstanding  of  the  role  of  qualification  could 
naturally  lead  to  misuse  of  the  program  in  government 
procurements . 

G.   THE  1968  DOD  STUDY  OF  QPLS 

1.  Introduction 

In  1965  the  Assistant  Secretary  of  Defense  for 
Installations  and  Logistics  directed  that  an  in-depth  study 
be  undertaken  to  ascertain  the  reasons  for  the  increase  of 
QPLs  and  to  determine  the  following: 

a.  Are  Qualified  Products  Lists,  in  fact,  necessary? 

b.  If  so,  are  they  required  to  the  extent  of  current 
coverage? 

c.  Are  the  current  criteria  governing  QPLs  meaningful  and 
realistic? 

d.  What  changes  are  necessary  to  more  effectively  operate 
and  control  the  qualification  process?   [Ref.  6:p.  4] 

2 .  Findings 

The  Departments  of  the  Army,  Navy  and  Air  Force  were 
directed  to  submit  responses  on  specifications  ranging  in 
age  from  as  much  as  18  years  to  those  issued  within  the  past 
six  months.   It  was  revealed  that  a  substantial  number  of 
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QPLs  were  issued  many  years  following  publication  of  the 
specification.  The  policy  at  that  time  called  for  the 
cancellation  or  revision  of  a  specification  where  a  QPL  had 
not  been  established  after  one  year  more  than  the  minimum 
period  of  time  estimated  for  actual  examination  and  testing. 
In  some  cases  specifications  were  as  much  as  ten  years  old 
before  a  QPL  was  issued.  This  indicated  qualification  was 
not  always  successful  in  eliminating  delay  in  delivery  of 
products.  Of  the  total  specifications  submitted,  695  had  no 
products  qualified  to  them.  There  were  747  QPLs  for  which 
only  one  producer  was  qualified.  The  military  departments 
indicated  that  on  55  documents,  qualification  had  been 
waived  at  least  once.  At  least  65  specifications  required 
100%  duplication  of  qualification  tests  during  product 
delivery  inspections.  Relative  to  the  justification  for 
imposing  qualification  in  a  general  specification,  62.2%  of 
the  documents  cited  time  required  to  conduct  compliance 
testing  would  unduly  delay  delivery  of  the  products  being 
purchased.  However,  there  appeared  to  be  no  pattern  for 
reviewing  justifications.  The  statistics  compiled  for  this 
study  showed  there  was  no  indication  that  time  is  in  fact 
being  saved  by  imposing  qualification  requirements  in  the 
specification.   [Ref.  6:p.  5] 

A  small  group  of  specifications,  within  a  10-20  year 
age  group,  maintained  QPLs  containing  anywhere  from  20  to 
more  than  100  different  qualified  products,  and  from  just  as 
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many  qualified  sources.  This  raised  the  question  as  to 
whether  the  product  in  question  had  not  stabilized  to  a 
point  where  it  is  a  common  item  or  material,  available  from 
anyone  within  the  given  industry.  The  elimination  of  quali- 
fication in  these  cases  should  have  been  considered. 

Additional  observations  and  questions  were  brought 
out  by  the  1968  study.  On  many  specifications,  the  military 
departments  maintained  that  they  need  assurance  of  quality 
prior  to  contract  award  so  as  to  guarantee  safety  in  flight. 
But  what  guarantee  is  inherent  in  a  product  that  has  been 
produced  one  time  by  a  given  manufacturer?  Does  this 
qualified  product  assure  quality  in  production?  Obviously 
the  answer  is  no.  Quality  if  controlled  only  by  appropriate 
measures  and  examination  taken  and  made  during  production 
and  it  is  the  good  specification  that  assures  this  quality 
through  its  requirements  and  conformance  testing.  It  was 
concluded  that  the  then  current  criteria  for  justification 
of  qualification  requirements  be  reconsidered.  The  study 
determined  that  the  time  required  for  testing  in  connection 
with  production  would  unduly  delay  delivery  of  the  supplies 
being  purchased  and  the  costs  of  repetitive  testing  would  be 
excessive,  were  the  only  real  justification  for  specifica- 
tions to  contain  qualification  requirements. 

The  findings  of  the  study  raised  the  question  why 
were  there  747  specifications  wherein  the  QPL  contained  one 
product,  available  from  one  source?   It  was  determined  there 
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must  be  many  other  reasons  why  additional  producers  have  not 
chosen  to  qualify  their  products  or  upgrade  their  products 
to  meet  the  requirements  of  the  specification.  Questions 
were  raised.  Is  one  manufacturer  being  favored  because  he 
exceeds  all  others  and  the  extra  advantage  of  a  second 
source  is  really  nor  necessary?  Is  testing  prescribed  in 
such  a  manner  that  to  comply,  producers  would  be  priced  out 
of  the  market?  Do  others  maintain  that  their  reputation  is 
such  that  their  products  will  be  bought  and  used  by  govern- 
ment prime  contractors  irrespective  of  whether  they  have 
qualified?  The  study  felt  these  questions  should  be 
explored  as  well  as  QPLs  with  no  producers  for  a  reasonable 
period  after  issuance,  must  be  questioned. 

The  55  documents  wherein  the  departments  indicated 
granting  of  waivers  to  qualification  requirements,  raised 
the  question  as  to  the  reason  for  the  waivers,  the  validity 
of  the  requirements,  and  the  need  for  qualification  in  the 
specification.  It  appeared  a  qualification  requirement 
should  be  questioned  if  waiver  becomes  necessary. 

During  the  1968  study,  associations  representing  a 
cross-section  of  American  industry  were  asked  to  submit 
comments  and  suggestions  relative  to  qualification  approval 
and  the  administration  of  the  process.  Industry  associa- 
tions in  general  agreed  that  the  qualification  process  has  a 
place  in  military  specifications.  The  following  is  a 
summary  of  their  1966-1967  comments: 
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a.  Qualification  has  merit  where  products  are  relatively 
new  and  for  which  little  experience  has  been 
established. 

b.  In  some  cases  QPLs  are  over  twenty  years  old  identify- 
ing products  now  obsolete  and  no  longer  required. 

c.  The  time  lapsed  between  approval  of  a  specification 
and  issuance  of  the  QPL  is  too  lengthy. 

d.  Identification  of  the  QPL  to  specific  revision  of  the 
specification  is  lacking. 

e.  QPLs  do  not  reflect  up-to-date  and  available  products 
in  all  cases  and  time  limits  for  review  of,  and/or 
requalif ication  of  products  is  not  adhered  to  or  is 
lacking. 

f.  Many  non-critical  items  require  qualification,  a 
decided  waste  of  effort  and  money. 

g.  Test  date  and  detailed  test  description  is  not  readily 
available  to  potential  users. 

h.  Once  a  product  is  qualified  and  placed  on  the  QPL,  the 
process  of  removal  for  good  cause  is  difficult,  almost 
impossible. 

i.  The  QPL  serves  as  a  useful  buyer's  guide  to  products 
that  have  been  qualified  once  to  a  published 
specification. 

j .  Retain  the  qualification  process  and  strengthen  the 
policies  and  procedures  that  govern  it. 

k.  Standardize  among  the  departments  relative  to  the 
number  of  qualification  reports  required  of  a  vendor. 

These  comments  represented  a  cross-section  of 
industry  associations  as  expressed  by  both  users  and 
manufacturers  of  products  contained  in  QPLs.   [Ref.  6:p.  7] 

As  the  comments  indicate,  manufacturers  were  essen- 
tially using  the  QPL  as  a  buyer's  guide.  However,  to  attain 
a  degree  of  quality  assurance,  some  contractors  relied  on 


29 


pre-testing  and  source  inspection  for  the  selection  of  a 

vendor.   As  stipulated  in  DOD  4120. 3M, 

inclusion  of  a  product  on  the  QPL,  .  .  .  does  not  in  any 
way  relieve  the  original  equipment  manufacturer  (OEM)  of 
his  contractual  obligations  to  ensure  that  delivered 
products  (including  the  qualified  products  used  in  the 
equipment)  comply  with  all  specification  requirements. 

Implicit  in  the  comments  is  that  many  OEMs  maintain  their 

own  data  on  vendors  products,  applied  their  own  testing 

programs  in  order  to  fulfill  their  contractual  obligations. 

OEM's  maintained  that  there  is  a  need  to  know  where 
on  the  manufacturing  experience  curve  a  given  vendor 
qualified  his  product.  In  critical  applications,  where 
reliability  is  an  important  factor,  this  becomes  extremely 
important  in  determining  appropriate  rescreening  procedures. 
At  that  time  the  government  was  obligated  to  protect  the 
information  received  from  manufacturers  unless  permitted  to 
do  otherwise.  The  study  recommended  that  this  policy  be 
reevaluated. 

Some  vendors  agreed  that  qualification  requirements 
are  necessary  and  that  the  process  is  the  fundamental 
cornerstone  of  an  efficient  procurement  management  system. 
Implied  in  these  comments  is  the  reasoning  that  the  QPLs  are 
the  means  by  which  "garage  operations,  profiteers,  disrepu- 
table producers,  and  the  like"  are  kept  from  government 
business.  However,  this  reasoning  was  never  intended  as  a 
justification  for  qualification  and  is  embodied  in  other 
federal  acquisition  regulations.   [Ref.  6:p.  9] 
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3 .   Recommendations 

The  study  concluded  that  the  qualification  process 
is  a  legitimate  procurement  technique,  but  recommended  the 
following: 

a.  All  military  departments  should  review  specifications 

containing  qualification  requirements  to  ensure 
conformance  with  current  criteria,  policies  and  proce- 
dures. The  elimination  of  QPLs  or  the  revision  of  QPL 
requirements  would  be  appropriate,  wherein  it  has  been 
revealed  that: 

(1)  qualification  tests  are  identical  to  conformance 
tests  (test  confirming  delivered  products  confor- 
mance to  specifications) , 

(2)  production  lead  time  is  at  least  twice  that  of 
qualification  test  time, 

(3)  no  products  are  qualified  to  the  specification 
and  the  document  is  in  force  for  one  year  more 
than  the  minimum  period  of  time  estimated  for 
actual  testing,  and 

(4)  qualification  has  been  waived  at  least  once  in 
the  past  year. 

b.  DOD  should  coordinate  the  following  with  a  goal 
towards  changing  current  practices  and  a  strengthening 
of  the  qualification  process: 

(1)  The  only  criteria  for  justifying  qualification 
should  be  the  time  required  for  testing  in 
connection  with  production  would  otherwise  unduly 
delay  delivery  of  the  supplies  being  purchased 
and  the  costs  of  repetitive  testing  would  be 
excessive. 

(2)  Rewrite  Chapter  IV  of  the  Defense  Standardization 
Manual  to  reflect  1.  above  and  place  the  follow- 
ing requisite  on  qualifying  activities: 

(a)  provide  for  requalif ication  requirements  in 
all  documents  calling  for  qualification  on  a 
periodic  basis, 

(b)  provide  evidence  of  requalif ication  on  pub- 
lished QPL, 
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(c)  expedite  publication  of  QPL  after  issuance 
of  specification, 

(d)  provide  enforced  criteria  for  removal  of 
deficient  suppliers  from  the  QPL, 

(e)  provide  an  accelerated  communication  system 
with  QPL  users  in  identification  of  all 
additions,  changes  and  removals, 

(f)  establish  periodic  time  limits  for  reviewing 
qualification  justification, 

(g)  establish  periodic  time  limits  for  reviewing 
QPLs  with  a  view  toward  elimination, 

(h)  establish  procedures  that  provide  specifica- 
tion managers  with  waiver  information,  and 

(i)  consider  means  for  making  qualification  data 
available  to  prospective  users.   [Ref.  6:p. 

11] 
The  preceding  recommendations  were  intended  to  give 

more  meaning  to  the  qualification  process  and  provide  a  more 

efficient  means  of  administering  the  program. 

H.   THE  1980  DOD  STUDY  OF  QPLS 
1 .   Background 

In  July  1979,  the  Director  of  the  Defense  Material 
Specifications  and  Standards  Office  tasked  its  staff 
engineer,  A.  Douglas  Reeves,  with  examining  DOD's  policies 
and  procedures  governing  the  establishment  and  use  of  QPLs. 
The  exact  nature  of  events  leading  to  this  study  is  not 
known.  However,  the  scope  was  intended  to  be  broad,  i.e., 
does  the  DOD  have  a  need  for  QPLs?  The  study  utilized 
readily  available  information  and  statistics.  Surveys  and 
data  gathering  exercises  were  intentionally  avoided.    The 
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report  was  intended  to  be  a  catalyst  to  allow  concerned 
parties  the  opportunity  to  refute  conclusions  and  recommen- 
dations and  provide  appropriate  data.  After  receipt  and 
evaluation  of  comments,  a  final  course  of  action  was  to  be 
defined  and  pursued.  The  following  seven  paragraphs  contain 
findings  with  noted  weaknesses  of  the  then  current  operation 
of  the  QPL  program  with  appropriate  recommendations  made  by 
Reeves  in  his  preliminary  report. 
2 .   Findings 

The  then  current  management  of  QPLs  was,  in  many 
area,  not  as  strong  as  it  should  have  been.  Good 
documentation  on  the  need  for  existing  QPLs  was  lacking. 
Periodic  review  requirements  should  be  strengthened. 
Statistics  were  typically  not  available  at  various  levels  of 
management  as  to  numbers  of  QPLs  with  zero  or  one  source. 
Reeves  felt  visibility  and  attention  needed  to  be  provided 
to  QPLs  in  each  commodity  class  on  a  continuing  basis.  One 
mechanism  which  cold  play  a  strong  role  in  this  regard  is 
the  Federal  Stock  Class  program  analysis.  Reeves  suggested 
that  this  annual  report  could  document  the  need  for  QPLs  in 
the  class  and  provide  effective  management  statistics  on  the 
qualification  program.  Examples  are:  statistics  as  to  the 
number  of  QPLs  in  each  commodity  area,  ratios  of  specifica- 
tions with  QPLs  versus  those  without,  identification  of  QPLs 
with  zero  or  one  source  and  the  length  of  time  the  situation 
has  existed.    Also  the  currency  of  QPLs  and  estimates  of 
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acquisition  activity  should  be  determined.  The  class 
assignee  activity  would  be  assuming  a  management  function 
with  regard  to  QPLs.   [Ref.  8:p.  4] 

During  the  late  197  0s  few  new  QPLs  were  being 
approved  in  the  Defense  Standardization  Program.  However, 
products  could  be  added  to  existing  QPLs  without  the  same 
levels  of  approval.  As  a  result,  many  new  products  were 
added  to  "general  type  specification"  QPLs  while  the  number 
of  QPLs  did  not  increase.  Strong  documentation  and  approval 
rules  were  recently  mandated  when  a  QPL  was  prepared,  but 
the  same  degree  of  scrutiny,  extensive  evaluation,  and 
justification  did  not  apply  to  perpetuation  of  a  QPL. 
Reeves  felt  that  the  requirement  for  qualification  to  be 
reviewed  at  intervals  not  greater  than  two  years  is 
necessary  and  should  be  thorough.  Greater  definition, 
controls,  and  visibility  should  be  provided  to  the 
discipline  of  qualification  when  it  is  applied.  Reeves 
recommended  that  the  DOD  should  initiate  an  intense  program 
to  review  qualification  procedures  in  each  commodity  area. 
Detailed  justification  should  be  developed  for  all  existing 
QPLs.  Many  should  be  cancelled.  Specifications  which 
require  QPLs  should  be  justified  as  absolutely  necessary. 
[Ref.  8:p.  14] 

Some  QPLs  existed  with  vendors  who  had  not  been 
retested  in  ten  years.  Many  QPLs  existed  with  zero  or  one 
source,   while   others  were   not  being  actively  procured. 
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These  needed  to  be  reviewed,  updated  or  cancelled, 
especially  those  involving  items  not  being  repetitively 
procured.  Reeves  also  perceived  a  reluctance  to  cancel  QPLs 
due  to  expected  pressure  from  those  who  had  borne  the 
expense  of  qualification  and  could  expect  to  find  their 
competition  increased.  This  problem  needed  to  be  addressed. 
[Ref.  8:p.  15] 

Reeves  found  that  DOD  QPL  policies  and  system  opera- 
tions varied  as  to  intensity  of  QPL  management  depending  on 
the  item  in  question  and  the  government  personnel  involved. 
Resource  problems  caused  many  of  the  limitations  and 
difficulties  in  the  operation  of  the  present  system.  QPLs 
should  only  be  established  and  maintained  when  adequate 
resources  are  available  to  provide  necessary  updates, 
manufacturer  recertif ications,  and  proper  program 
management.  New  QPLs  should  be  published  at  specific 
intervals  with  dates  of  latest  manufacturer  requalif ication. 

Reeves  discovered  that  under  the  then  current 
program,  if  a  general  specification  required  qualification, 
all  items  covered  by  a  detailed  specification  also  have 
qualification  associated  with  them.  Reeves  believed  that 
qualification  procedures  should  be  used  only  for  high  usage 
items  and  not  as  a  requirement  on  lower  usage  items,  i.e., 
items  not  procured  in  large  quantities.  Detailed  rules  for 
qualification  should  be  developed  for  each  commodity  area. 
Market  conditions,  item  criticality,  item  characteristics, 
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and  testing  requirements  vary  sufficiently  to  warrant 
different  system  structures  and  control  techniques. 

It  had  been  argued  that  by  mandating  the  use  of  QPLs 
a  contractor  is  forced  to  utilize  quality  components.  But 
Reeves  stated  that  perhaps  the  most  important  constraint  for 
the  contractor  for  the  overall  system  being  produced  comes 
through  environmental/life  testing  conducted  as  part  of  the 
system  demonstration.  This  would  seem  to  be  a  forcing 
function  insuring  that  contractors  do  what  is  necessary  to 
control  product  quality  and  specification  compliance  by 
their  suppliers.  System  contractors,  where  they  represent 
predominant  QPL  users,  should  have  the  opportunity  to 
provide  strong  inputs  into  both  the  nature  of  QPLs  and  their 
individual  justification.   [Ref.  8:p.  15] 

Reeves  commented  that  QPLs  can  limit  competition  and 
can  be  expected  to  increase  acquisition  costs.  The  expense 
of  qualification  can  be  significant  and,  since  no  guarantee 
of  obtaining  a  contract  and  amortizing  these  costs  exists, 
can  have  a  negative  effect  on  the  potential  number  of 
suppliers.  This  can  be  particularly  troublesome  in  a 
limited  source  or  sellers  market.  This  was  determined  to  be 
another  factor  a  preparing  activity  should  consider  in 
determining  the  need  for  a  QPL.  Increased  participation  by 
the  govern-ment  in  sharing  the  costs  of  testing  in  this 
situation  when  a  QPL  is  deemed  necessary,  may  be  desirable. 
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3 .   Conclusions  of  the  1980  POD  Study 

The  actual  significance  and  integrity  of  a  QPL  is  a 
function  of  the  many  factors  comprising  the  detailed 
operation  of  the  system,  such  as  the  time  allowed  to  elapse 
between  recertif ication,  degree  of  enforcement,  and  the 
relationships  between  tested  units  and  delivered  production 
units.  Over  the  range  of  commodity  items  covered  by  QPLs, 
there  is  a  great  variation.  Implicit  in  the  program  is  the 
idea  that  QPLs  assume  functions  in  the  Quality  Assurance 
arena.  Where  that  may  be  appropriate,  program  features, 
such  as  enforcement  techniques,  comprehensive  feedback 
mechanisms,  and  auditing  and  testing  requirements  should  be 
structured  to  allow  a  strong,  positive  reliance  on  items 
procured  from  QPL  sources.  Inherent  in  the  program  should 
be  continuing  monitoring  features  to  assure  and  demonstrate 
cost  effectiveness  of  these  additional  quality  assurance 
techniques. 

Difficulty  in  developing  qualified  sources  was  iden- 
tified as  a  problem  for  low  volume  item  specifications. 
Reeves  suggested  these  problems  would  best  be  handled  by  not 
requiring  qualification  for  low  volume  items.  Qualifica- 
tion, in  such  instances,  was  not  an  appropriate  acquisition 
technique.  Particular  difficulty  existed  in  trying  to 
rationalize  the  mandatory  nature  of  QPLs  as  applied  to  the 
cumbersome  process  of  obtaining  waives  and  trying  to  justify 
qualified  products  with  zero  or  one  qualified  source.   First 
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article  inspection  and  other  methods  of  assuring  necessary 
demonstration  of  requirements  would  allow  better  flexibility 
in  matching  costs  and  risks.  The  specification  preparer 
needs  to  be  more  actively  involved  in  the  actual  acquisition 
environment.  He  should  have  a  keen  appreciation  of  the 
acquisition  situation  (number  of  acquisitions,  quantities, 
costs,  percentage  contractor  versus  direct  government  acqui- 
sitions and  unique  acquisition  problems) .  Proper  feedback 
mechanisms  between  the  procuring  activity  and  the  specifi- 
cation preparing  activity  should  be  developed. 

4 .   Final  Recommendation  of  the  1980  POD  Study 

The  1980  report  suggested  that  the  criteria  for 
establishing  QPLs  should  be  expanded.  QPLs  are  justifiable 
only  when  they  enhance  product  quality,  reduce  acquisition 
risks,  or  avoid  otherwise  excessive  costs  or  delays  in 
procurement.  If  the  government  could  afford  complete 
verification  of  all  specification  requirements,  including 
qualification  testing,  in  all  acquisition  situations,  the 
need  for  qualification  procedures  would  be  moot.  However, 
some  trade-offs  in  quality,  risk,  and  cost  must,  on  occa- 
sion, be  made. 

The  1980  report  suggested  that  the  set  of  circum- 
stances which  should  prompt  consideration  of  establishing  a 
QPL  should  be  as  follows: 

a.  An  item  is  procured  repetitively. 

b.  The  unit  dollar  value  of  the  items  is  relatively  low. 
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c.  Some  relatively  complex/ expensive  tests  are  required 
to  demonstrate  compliance  with  specification 
requirements . 

d.  Demonstration  of  compliance  with  qualification 
requirements  is  not  critical  in  terms  of  mission  or 
safety. 

When  these  conditions  do  not  exist,  a  QPL  would  not 
generally  be  warranted.  If  qualification  is  intended  to 
include  quality  assurance  functions,  criterion  c.  can  be 
replaced  or  supplemented  by  the  following:  "extensive 
supplier  control  and  inspections  are  necessary  to  demon- 
strate compliance  with  specification  requirements."  [Ref. 
8:p.  16] 

Could  QPLs  be  eliminated?  In  an  absolute  sense, 
Reeves  stated,  the  answer  was  yes.  Other  techniques  are 
available  which  could  be  substituted  in  QPLs  were 
eliminated.  Pre-award  surveys  which  verify  contractors' 
technical  capabilities,  first  article  testing,  and  varia- 
tions of  these  approaches  could  fill  the  void  created  in 
QPLs  were  eliminated  entirely,  although  not  without  cost  or 
risk  ramifications.  However,  Reeves  felt  that  the  qualifi- 
cation process  should  be  retained  as  an  option  for  inclusion 
in  specifications.  The  philosophy  of  QPLs  has  a  legitimate 
basis  predicated  on  the  complex  acquisition  environment  in 
which  it  is  used.  There  are  both  apparent  and  real 
contradictions  in  and  limitations  to  the  way  the  DOD  uses 
QPLs;  but  the  basic  concept  has  application.  They  should  be 
used  only  when  other  techniques  would  result  in  unsuitable 
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risks  or  costs  and  when  benefits  commensurate  with  costs  can 
be  demonstrated.  Increased  use  of  QPLs  should  not  be 
encouraged.  References  to  "intangible  benefits"  of  the  QPL 
system  was  made  on  occasion.  These  "benefits"  will  be 
discussed  later. 

5.   Summary  of  the  198  0  POD  Study 

Reeves  felt  a  comprehensive  review,  assessment,  and 
development  of  alternative  QPL  policies  and  procedures  in 
selected  commodity  areas  should  be  initiated.  Service 
groups,  with  interaction  with  affected  industry  segments, 
should  be  convened  for  this  function.  Qualification  is  a 
legitimate  acquisition  option.  The  process  is,  however, 
misapplied  and  could  be  improved.  Lack  of  program 
intensity,  as  a  result  of  inadequate  resources,  resulted  in 
many  of  the  problems  discussed.  When  the  report  was 
published,  Reeves  believed  that  the  recommendations  would 
stimulate  discussion,  result  in  a  redefinition  of  objec- 
tives, generate  momentum  for  change,  and  ultimately 
strengthen  the  program. 

I.   SENATE  HEARINGS  BEFORE  THE  COMMITTEE  ON  SMALL  BUSINESS 
1 .   Background 

A  recent  event  has  affected  the  QPL  program.  This 
was  the  Hearings  before  the  Committee  on  Small  Business  of 
the  United  States  Senate  held  on  April  6  and  April  12,  1984, 
discussing  the  Small  Business  Competition  Enhancement  Act  of 
1984.   The  majority  of  the  testimony  concerned  the  need  for 
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greater  government  effort  toward  promoting  the  open  competi- 
tion of  small  business  in  the  federal  procurement  of  spare 
parts.  During  the  course  of  proceedings  the  QPL  program  was 
repeatedly  mentioned.  A  summary  of  the  main  points  of 
discussion  and  opinions  follow. 

The  bill  had  two  main  goals.  The  first  goal  was  to 
increase  small  business  participation  in  the  federal 
procurement  process,  thereby  reducing  costly  noncompetitive 
procurements.  The  second  was  to  broaden  our  nation's  indus- 
trial base  for  civilian  and  defense  procurements.  The  bill 
would  require  the  procuring  agency  to: 

a.  justify  in  writing  the  need  for  prequalif ication, 

b.  formalize  and  make  known  the  standards  required  to 
prequalify, 

c.  provide  opportunities  for  those  desiring  to  qualify, 
and 

d.  provide  test  and  evaluation  services  at  no  cost  to  a 
small  business  when  insufficient  qualified  suppliers 
are  available.  All  failures  to  qualify  would  be 
reported  back  to  the  business  concern  with 
justification. 

Free  testing  and  evaluation  services  for  small 
businesses,  where  additional  sources  are  deemed  necessary, 
was  designed  to  lower  product  cost,  expand  industrial  base, 
and  allow  entry  of  newer  products  with  higher  quality.  Also 
at  issue  was  the  extent  to  which  procuring  agencies  should 
be  allowed  to  use  prequalif ication  of  sources  as  a  procure- 
ment technique. 
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The  testimony  given  can  be  broken  down  into  three 

general  areas:   testimony  was  given  by  individuals  in  favor 

of  the  QPL  program,  those  individuals  wanting  the  program 

eliminated,   and  those  individuals  seeing  the  program  as 

having  value,  but  needing  changes.   Proponents  of  the  QPL 

program  stated, 

The  use  of  QPLs  .  .  .  serves  an  extremely  important 
purpose.  There  are  many  parts  and  components  of  weapons, 
ships,  and  aircraft  systems  that  are  not  only  critical  to 
the  proper  operation  of  these  systems,  but  also  critical 
to  the  safety  of  their  crews  and  operators.  Maintaining  a 
list  of  prequalified  products  is  one  way  of  assuring  only 
acceptable  items  will  be  considered.   [Ref.  9:p.  75] 

J.  Brosnan  of  the  National  Security  and  International 
Affairs  Division  of  the  U.S.  General  Accounting  Office 
stated,  "QPLs  are  needed  .  .  .  problems  lie  in  the  abuse  of 
the  system."  [Ref.  9:p.  181]  W.  Adams  Jr.  from  the 
National  Security  Industrial  Association  commented, 
".  .  .we  believe  that  there  is  a  valid  need  for 
prequalif ication  requirements  under  specific  circumstances 
as  determined  by  government  agencies,  especially  the 
Department  of  Defense."  [Ref.  9:p.  394]  S.  Evans, 
Assistant  Administrator  for  Procurement  at  NASA,  pointed 
out,  "There  are  often  market  situations  where  only  a  few 
firms  may  possess  the  resources  and  technical  expertise  to 
manufacturer  items  that  meet  the  exacting  needs  of  the  space 
environment."  [Ref.  9:p.  222]  Qualified  products  'S'  lists 
or  space  lists,  are  commonly  issued  by  NASA.  The  concen- 
trated support  for  QPLs  came  from  the  Electronics  Industries 
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Association  Components  Group.   H.J.  Rowe,  representing  the 

group,  stated, 

The  (proposed)  restrictions  on  qualification  or  prequali- 
fication  .  .  .  are  of  concern  to  us  in  that  we  see  a 
strong  potential  for  lowering  the  quality  of  the  products 
and  systems  that  comprise  our  industrial  mobilization 
base.  .  .  .  Qualification  is  so  important  for  electronic 
components  that  a  domestic  and  world-wide  system  has  been 
developed  for  commercial  and  industrial  components.  It  is 
essential  for  electronic  systems  that  performance  and 
reliability  is  verified.   [Ref.  9:p.  231 

He  felt  that  a  history  of  reliability  and  quality  has  been 

established  through  the  combined  efforts  of  industry  and 

government  procuring  agencies.    The   specifications   and 

standards  so  derived  represent  the  culmination  of  years  of 

effort  to  provide  a  quality  product  at  minimum  cost  to  the 

government.   Even  one  opponent  of  the  program,  R.  Ludwig, 

speaking  on  behalf  of  the  National  Federation  of  Independent 

Business,  admitted,  "there  are  some  times  when  you  must  have 

a  qualified  manufacturer  make  a  part,  I  think  that  falls 

more  rightly  into  electronic  components,  for  example,  where 

a  component  has  to  do  something  other  than  just  be  there." 

During  the  testimony  the  most  vocal  opponent  of  the 

QPLs  was  B.  Hahn,  Manager  of  Government  Affairs  with  the 

National  Tooling  and  Machining  Association.    He  was  for 

elimination  of  the  current  QPL  program  for  the  following 

reasons: 

a.   The  program  has  evolved  to  the  point  where  its  main 

function  is  to  prevent  competitive  procurements.  Hahn 
stated  that  of  the  3.9  million  parts  in  DOD's  spare 
parts  inventory,  only  7%,  or  approximately  275,000, 
are  procured  through  open  competition.  Most  of  the 
remaining   93%,   or   approximately   3,625,000   parts, 
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require  bidder  and/or  product  prequalif ication  before 
a  new  source's  bid  will  be  considered.  Some  of  the 
many  thousands  of  "critical"  items  on  these  parts 
lists  are  floor  wax,  automotive  grease,  filing 
cabinets,  shipboard  loudspeakers,  lightbulbs,  and 
spark  plugs. 

b.  Prequalif ication  is  used  as  a  means  to  deprive  small 
business  concerns  of  their  statutory  right  to  have  SBA 
make  the  final  determination  of  their  capability  to 
perform  under  its  certificate  of  competency  program. 
Normally  the  SBA  has  the  final  determination  on 
whether  a  small  business  concern  has  the  capability  to 
produce  quality  products  in  sufficient  quantities  for 
specific  government  acquisitions,  however,  no  business 
concerns  can  bid  for  prequalif ication  products  if  they 
are  not  first  qualified. 

c.  Prime  contractors  (OEMs)  make  the  decision  whether  a 
component  should  be  prequalif ied,  thus  restricting 
suppliers. 

d.  Quality  control  cannot  be  achieved  through  prequalif i- 
cation.  Other  quality  control  techniques  are  far  more 
effective  and  should  be  utilized. 

e.  DOD's  specification  data  storage  and  retrieval  capa- 
bilities are  close  to  nonexistent.   Without  knowledge 
of  the  standards   required  to  qualify,   prospective 
competitors  are  locked  out.   B.  Hahn  supported  this 
reason  with  the  following  example: 

A  group  of  NTMA  members  and  staff  were  invited  to  Tinker 
AFB,  reputedly  one  of  their  more  efficient  procurement 
activities,  last  year  by  Air  Force  Secretary  Verne  Orr.  A 
very  large  portion  of  the  spare  parts  procured  there  fall 
into  one  of  these  five  categories: 

(1)  We  have  the  data,  but  it's  been  reproduced  so  many 
times  that  it  has  become  illegible  so  we  have  to  buy 
sole  source. 

(2)  We  don't  have  the  data  and  we  don't  know  where  it  is 
so  we  have  to  buy  sole  source. 

(3)  We  have  the  data  and  it  is  complete,  but  we  don't 
know  whether  it's  proprietary  because  we  destroyed 
the  original  contract  which  tells  us  which  data  we 
own  so  we  have  to  buy  it  sole  source. 
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(4)  We  have  part  of  the  data  and  it's  legible.  If  we  can 
ever  find  the  missing  pages,  maybe  we  can  stop  buying 
sole  source. 

(5)  The  data  is  marked  proprietary  and  since  we  do  not 
have  the  qualified  personnel  to  review  proprietary 
claims,  we  assume  that  it  must  be  proprietary  so  we 
have  to  buy  the  part  sole  source.   [Ref.  9:p.  420] 

Other  comments  brought  up  at  the  hearings  criticiz- 
ing the  system  included:  it  creates  "astronomical"  adminis- 
trative expenses  to  the  small  businessman  with  no 
assurance/indication  of  potential  return  on  his  investment. 
QPLs  have  been  used  to  favor  "friends"  or  to  effectively 
limit  competition.  As  a  practical  matter  QPLs  are 
anticompetitive  because  they  restrict  the  number  of  partici- 
pants in  procurements  before  solicitations  are  even  issued. 
Such  lists  can  thus  be  viewed  to  operate  as  de  facto  respon- 
sibility determinations,  or  nonresponsibility  determina- 
tions, as  the  case  may  be.  [Ref.  9: p.  3  63]  More  products 
are  on  QPLs  than  need  to  be.  More  resources  are  needed  for 
effective  administration  of  the  program.  When  used  to 
insure  quality  control  standards,  the  QPL  program  is 
redundant  in  its  efforts.  Other  manufacturers  have 
commented,  "It's  not  worth  it.  It's  too  much  of  a  hassle. 
You  can't  open  the  door.  There  is  too  much  red  tape,  too 
bureaucratic,  too  complicated."   [Ref.  9:p.  355] 

Many  recommendations  were  made  on  how  to  improve  the 
QPL  program,  allow  for  more  competition,  obtain  better 
quality  parts,  and  lower  unit  costs.  Some  of  these 
included:   criteria  for  prequalif ication  standards  must  be 
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clarified  and  made  easily  available  to  prospective  competi- 
tors. Specifications  should  be  more  readable  and  available. 
Prequalif ication  requirements  should  be  automatically 
challenged  so  that  they  could  be  employed  only  in  situations 
where  a  legitimate  need  is  found.  Qualified,  independent 
consultants,  who  are  knowledgeable  and  experienced,  should 
determine  the  need  for  the  prequalif ication  of  a  product, 
not  the  OEM.  QPLs  must  not  become  so  encompassing  that  it 
includes  products  that  have  no  critical  function  in  a  major 
system.  To  meet  the  dual  concerns  of  providing  the  products 
in  a  timely  manner,  as  well  as  insuring  that  the  product 
fully  meets  the  requirement,  other  procurement  techniques 
can  be  used,  such  as  preaward  surveys,  simply  technical 
prequalif ication  reviews,  responsibility  determinations,  and 
first  article  testing.  Use  of  QPLs  and  other  quality 
assurance  techniques  could  then  be  expressly  sanctioned  as  a 
means  of  assuring  quality  of  "genuinely"  critical  parts. 
[Ref.  9:p.  82]   R.  Ludwig  adds, 

Government  should,  with  its  vast  and  comprehensive 
resources,  be  able  to  evaluate  bidders  and  components 
without  resorting  to  a  QPL.  After  all,  there  is  no 
guarantee  that  a  previously  qualified  supplier  will 
continued  to  supply  high  quality  parts. 

Ludwig  specifically  recommends  a  "qualified  manufacturer's 

list" — a  list  that  qualifies  a  company  to  make  a  class  of 

parts  or  a  type  of  manufacture  rather  than  trying  to  qualify 

that  supplier  to  make  an  individual  part.    [Ref.  9:p.  85] 

It  was  also  recommended  that  the  DOD  update  its  data  storage 
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and  retrieval  system.  Such  an  update,  it  was  estimated, 
would  save  the  government  $5-6  billion  a  year  in  spare  parts 
procurement.   [Ref.  9:p.  4  62] 

Lastly,  it  was  brought  out  that  the  DOD  and  industry 
work  together  to  assure  that  specifications  applied  to  new 
major  system  designs  are  necessary  and  cost  effective.  They 
should  be  tailored  and  not  become  excessive  for  the  intended 
applications.  Engineering  designs  should  continue  to  be 
justified  on  the  basis  of  a  valid  life-cycle  cost  savings  or 
as  a  response  to  a  safety  concern.   [Ref.  9:p.  416] 

J.   CHAPTER  SUMMARY 

This  chapter  discussed  briefly  the  history  of  the  QPL 
program.  It  covered  in  some  depth  two  DOD  studies  concern- 
ing the  application  of  the  qualification  process  in  govern- 
ment procurements.  The  chapter  also  covered  Senate  hearings 
concerning  the  ramifications  of  the  qualification  process  on 
a  cross-section  of  small  businesses.  Some  of  the  more 
important  points  brought  out  by  the  reports  and  hearings 
include: 

1.  The  qualification  process  is  important  as  an  alterna- 
tive acquisition  technique.  The  major  problems  stem 
from  an  abuse  of  the  program  applications.  This  is  a 
result  of  either  a  misunderstanding  of  qualification 
goals  or  a  misuse  of  the  process. 

2.  QPLs  must  be  periodically  and  thoroughly  reviewed  by 
the  responsible  agency.  Many  should  be  revised, 
updated  or  eliminated.  To  help  ensure  effective 
management  of  the  program,  adequate  resources  must  be 
made  available. 
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3.  The  qualification  process  should  be  tailored  to  the 
wide  variation  of  commodity  areas  covered  by  QPLs. 

4.  The  function  of  the  qualification  process  toward 
quality  assurance  is  not  well  defined.  Other  quality 
assurance  techniques  must  accompany  the  qualification 
testing.  However,  100%  duplication  of  testing  is 
considered  wasteful. 

5.  The  government  must  modernize  its  data  storage  and 
retrieval  capabilities  to  effectively  administer  the 
QPL  program  and  enhance  its  effectiveness. 

6.  Qualified,  independent  experts  should  determine  the 
necessity  of  the  qualification  process  in  specifica- 
tions. Justification  for  qualification  deserves  close 
review. 

7.  The  qualification  process  should  be  managed  so  as  not 
to  eliminate  competitive  bidding  for  government 
acquisitions. 

As  this  chapter  showed,  QPLs  have  been  the  focus  of 

much  attention,  criticism,  and  debate.   As  a  concept  QPLs 

have  a  purpose,  but  it  appears  that  the  execution  of  the 

program,   the  resources  devoted  to  the  program,   and  its 

applications   could   use   improvements.     There   are   many 

commodity  areas  the  QPLs  cover  and  many  perspectives  that 

can  be  taken  toward  the  program.   In  the  next  chapter  the 

current  QPL  program  will  be  briefly  covered,  as  well  as  its 

application  to  the  rapidly  advancing  technological  area  of 

electronics.    Current  perspectives  of  the  QPL  program  by 

electronic  contractors  and  subcontractors  will  be  discussed. 
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IV.   QUALIFIED  PRODUCTS  LISTS  AND  THE 
PROCUREMENT  OF  SEMICONDUCTORS 


A.   INTRODUCTION 

QPLs  cover  a  wide  range  of  commodity  areas.  It  would 
take  volumes  of  pages  to  analyze  the  application  of  the  QPL 
process  to  each  commodity  area.  Instead,  this  paper  will 
attempt  to  cover  the  strengths  and  weaknesses  of  the  QPL 
process  as  it  applies  to  the  electronics  sector  of  U.S. 
industry  and  in  particular  the  semiconductor  area.  The 
semiconductor  area  was  chosen  because  of  the  rapid  rate  of 
technological  advances  and  the  increasing  value  of  the  area 
to  military  weapon  systems.  The  QPL  process  as  it  applies 
to  semiconductors  is  also  the  subject  of  much  controversy  as 
to  its  appropriateness  and  effectiveness.  Details  of  this 
controversy  will  be  discussed  later  in  this  chapter. 

This  chapter  will  first  provide  some  basic  definitions 
of  semiconductors.  A  brief  history  follows  describing  the 
beginnings  of  government  procurement  of  semiconductors,  the 
Joint  Army-Navy  (JAN)  program,  and  current  policies  and 
problem  areas  in  the  acquisition  process.  Observations  and 
comments  of  contractors  and  manufacturers  in  the  semiconduc- 
tor industry  will  conclude  the  chapter.  The  final  chapter 
will  provide  an  analysis  of  the  QPL  process  in  the  semicon- 
ductor  industry   and   other   commodity   areas   in   general. 
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Recommendations  based  on  conductor  research  and  interviews 
will  then  be  provided. 

B.   DEFINITIONS 

The  semiconductor  industry  is  the  major  sub-sector  of 
the  electronic  components  industry  producing  active 
components  (i.e.,  components  which  modify  and  control 
electrical  signals  by  amplification,  switching  action  or 
modulation  of  the  signal  in  a  circuit) .  Semiconductor 
devices  can  be  distinguished  in  terms  of  their  functional 
breakdown  as  electronic  components.  Semiconductors  can  be 
either: 

1.  discrete  components:  devices  composed  of  a  single 
electronic  circuit  such  as  transistors,  rectifiers  and 
diodes , 

2.  integrated  circuits:  microprocessors,  which  can  be 
considered  as  a  computer  processor  on  a  single  chip 
and  memories,  which  store  information  in  the  form  of 
electrical  charges,  or 

3.  special  purpose  devices:  devices  not  categorized  in 
either  of  the  previous  two  functions.  [Ref.  10  Appen- 
dix A] 

The  fastest  growing  major  semiconductor  product  area  is 

integrated  circuits.   It  is  in  the  integrated  circuit  market 

where  the  greatest  efforts  are  being  made   in  terms  of 

research  and  development,  where  the  greatest  progress  has 

been  made  in  product  innovation,  and  which  has  had  the  most 

significant  impact  on  end-users  (i.e.,  government). 
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C .   BACKGROUND 

The  first  integrated  circuits  were  commercially  marketed 
in  1962.  During  this  time  integrated  circuit  technology  was 
driven  by  production  for  military  and  government  end-use 
products .  DOD  and  NASA  procurement  during  integrated 
circuits  early  development  offered  a  constant  level  of 
demand  for  integrated  circuits  at  premium  prices.  This 
served  to  widen  the  industrial  base  through  which  techno- 
logical innovation  in  design  and  production  of  integrated 
circuits  continued  to  advance.  In  1963  government  procure- 
ment accounted  for  95%  of  the  $4  million  market  for  inte- 
grated circuits  and  the  average  selling  price  of  each 
circuit  was  $50.  In  two  years  the  number  of  non-government 
integrated  circuit  uses  expanded  with  total  integrated 
circuit  production  reaching  $80  million.  Government 
procurement  dropped  to  7  5%  of  the  market.  The  average 
integrated  circuit  selling  price  had  fallen  to  $9.  [Ref. 
10:pp.  12-13] 

Government  procurement  had  encouraged  and  accelerated 
the  pace  of  technological  advancement  in  the  industry.  New 
manufacturers  came  into  existence  prompted  by  the  rapidly 
growing  market.  From  1965  through  1972  the  demand  created 
by  a  rapidly  expanding  commercial  electronics  and  computer 
equipment  market  propelled  technological  advancement  and 
market  growth  in  the  industry.  The  commercial  market  was 
now  the  driving  force  behind  innovation  and  production.   By 
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1972  government  purchases  had  dropped  to  below  25%  of  the 
$680  million  total  U.S.  production  of  integrated  circuits. 
From  1972  to  1985  the  demand  pull  from  the  newly  emerging 
commercial  mass  markets  created  a  move  towards  standardized 
circuit  functions.  Because  of  the  newly  intensified  inter- 
national competition,  integrated  circuit  prices  rapidly 
declined.  U.S.  manufacturers  shifted  production  processes 
to  cater  to  the  commercial  market,  producing  higher  volume, 
standardized  circuits  at  lower  prices.  Integrated  circuits 
found  new  applications  in  semiconductor  memories,  consumer 
products,  telecommunications,  and  in  microcomputers.  During 
this  period  military  procurements  of  semiconductors  once 
again  increased  and  became  even  more  vitally  important  to 
U.S.  national  defense  strategy.  With  the  major  defense 
systems  building  up,  semiconductor  sales  to  the  government 
grew  from  $247  million  in  1975  to  an  estimated  $1.3  billion 
in  1985  [Ref.  10:p.  18].  Despite  the  fact  that  the 
commercial  sector  of  the  business  remained  the  driving  force 
for  product  application,  military  sales  of  semiconductors 
was  growing  rapidly  enough  and  steadily  enough  so  that  by 
1985  U.S.  companies  were  again  giving  military  sales  serious 
attention.  To  signify  the  importance  of  semiconductors  in 
major  military  defense  systems,  the  Electronic  Industries 
Association  predicts  that  semiconductor  sales  for  government 
end-use  applications  will  increase  to  $3.1  billion  in  1980 
from  $1.16  billion  in  1984.    Over  this  same  period  the 
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electronics  portion  of  the  defense  budget  will  increase  by 
3.2%.   [Ref.  10:p.  vii] 

Semiconductor  products  play  a  vital  role  in  the  U.S. 
national  defense  and  will  likely  play  an  even  greater  role 
in  the  future.  Before  the  comments  of  semiconductor 
manufacturers  and  contractors  are  discussed,  the  current 
acquisition  regulations  will  be  introduced. 

D.   ACQUISITION  POLICY  AND  PROCEDURES 

During  the  birth  of  the  semiconductor  industry,  the 
military  and  other  government  agencies  were  the  driving 
forces  behind  its  technological  innovations  and  applica- 
tions. To  guard  against  defective  semiconductor  products  in 
government  end-use  items ,  the  government  and  the 
semiconductor  producers  introduced  a  system  of  testing  and 
production  requirements?  this  was  considered  a  sensible 
approach  to  quality  assurance. 

During  the  late  1960's,  as  volume  production  became 
possible,  a  new  revision  to  U.S.  government  semiconductor 
procurement  requirements  took  place.  In  1969  this  revision 
created  a  new  government  Qualified  Parts  List,  the  Joint 
Army-Navy  (JAN)  system.  This  system  was  created  to 
encourage  the  production  of  higher  volumes  of  products  for 
government  end-use  applications.  With  the  creation  of  the 
JAN  system,  additional  testing  and  production  requirements 
were  deemed  necessary  to  insure  an  acceptable  level  of 
quality  control.    The  general  Mil-M-38510  specification, 
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also  known  as  the  JAN  system,  is  made  up  of  a  general  speci- 
fication and  a  series  of  detailed  drawings  called  "slash 
sheets."  Mil-M-38150  identifies  all  design,  materials, 
finish,  tests,  and  qualification  requirements  for  integrated 
circuits.  Any  part  sold  with  a  JAN  marking  must  meet  over 
150  pages  of  requirements .  Specific  requirements  of  the 
slash  sheets  typically  contain  30  to  40  pages  of  performance 
specifications.   [Ref.  10:Appendix  B] 

Semiconductor  devices  used  in  military  applications  can 
also  be  purchased  through  three  other  DOD-approved  specifi- 
cation systems.   They  include: 

1.  MJ1-STD-883C:  basically  an  encyclopedia  of  test  and 
inspection  methods  to  be  used  on  integrated  circuits 
supplied  to  the  government.  It  also  defines  the 
screening  and  quality  conformance  test  flows  for  inte- 
grated circuits. 

2 .  Defense  Electronic  Supply  Center  (PESO  Drawings:  a 
drawing  system  that  is  not  as  rigid  as  the  JAN  system 
but  attempts  to  provide  a  standardization  mechanism 
for  description  of  part  performance  and  screening 
requirements.  DESC,  responsible  for  the  qualification 
and  administration  of  the  JAN  qualified  parts  lists, 
created  this  drawing  system  in  response  to  the  declin- 
ing usage  of  JAN-QPL  listed  products. 

3.  Source  Control  Drawings  (SCDs) :  a  custom-tailored 
description  of  a  commercially  purchased  part.  These 
include  the  contractor's  own  set  of  processing  proce- 
dures and  test  parameters,  as  long  as  they  satisfy 
certain  minimum  requirements.  SCDs  allow  contractors 
or  subcontractors  to  retain  parts  control,  to  design 
with  the  latest  technologies  and  to  meet  cost  objec- 
tives. However,  SCDs  tend  to  lead  to  a  proliferation 
of  different  part  types. 

Generally,  contractors  must  look  first  to  the  QPL  lists 

for  military  integrated  circuit  needs  before  resorting  to 

other  specifications,  such  as  DESC  drawings  or  their  own 
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source  control  documents.  The  use  of  SCDs  rather  than  JAN 
parts  greatly  increases  the  number  of  part  types  a  semicon- 
ductor manufacturer  must  process  for  contractors.  DESC 
estimates  the  number  of  national  stock  numbers  for 
semiconductor  devices  is  now  in  the  vicinity  of  80,000, 
while  the  number  of  integrated  circuit  designs  that  exist  in 
the  commercial  market  is  only  around  5,000.  [Ref.  11 :p.  66] 
Now  that  a  brief  introduction  to  the  history  of  semicon- 
ductor procurement  has  been  covered,  as  well  as  DOD-approved 
procurement  systems  for  semiconductors,  problems  in  the 
current  acquisition  system  will  be  addressed  from  the 
semiconductor  industry's  perspective.  In  October  of  1985 
the  Semiconductor  Industry  Association  (SIA)  prepared  its 
White  Paper  to  the  DOD.  It  addresses  government  procurement 
of  semiconductors.  Some  of  the  problem  areas  addressed  in 
that  paper  follow. 

E.   ISSUES  CONCERNING  GOVERNMENT  ACQUISITION  OF 
SEMICONDUCTORS 

These  are  some  of  the  problems  brought  out  by  the  SIA  in 

its   white   paper   concerning   government   procurement   of 

semiconductors . 

1.  Semiconductor  manufacturers  do  not  have  adequate 
representation  in  developing  semiconductor  procurement 
regulations  and  policies.  They  are  too  far  removed 
from  government  contracting  representatives. 

2 .  Government  continues  to  procure  semiconductors  with  a 
policy  that  results  in  devices  of  the  same  quality  and 
reliability  as  commercial  products,  but  at  a  higher 
cost. 
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3 .  Present  procurement  procedures  result  in  nonstandard 
products,  low  volume  lot  sizes,  massive  volumes  of 
inspection  and  testing  data  requirements,  and  utiliza- 
tion of  "sunset  technologies."  "Sunset  technology"  is 
technology  that  is  past  its  mature  growth  phase, 
bordering  on  obsolescence. 

4 .  The  present  government  procurement  system  encourages 
semiconductor  users  to  build  into  their  government 
systems  a  wide  range  of  different  part  types,  thus 
increasing  specification  documentation  as  well  as 
costs . 

5.  Standardization  requirements  emphasize  inspecting  for 
quality  rather  than  designing  in  quality. 

6.  Major  defense  systems  are  designed  for  operational 
lives  of  up  to  3  0  years  resulting  in  extended  semi- 
conductor life  cycles.  This  could  result  in  continued 
use  of  obsolete  technology  and  extremely  high  replace- 
ment costs. 

7.  Current  government  inspection  and  testing  requirements 
have  become  inappropriate  and  near  impossible  with  the 
advancement  of  integrated  circuit  technology. 

8.  Expensive  and  obsolete  integrated  circuit  packages  are 
still  required  by  government  specifications. 

9 .  The  JAN  program  and  other  government  semiconductor 
procurement  programs  are  highly  resistent  to  change 
and  advances  in  technology.  In  order  for  the  govern- 
ment to  obtain  the  latest  technology,  achieve  the 
highest  quality  and  reliability  at  the  best  price, 
government  procurement  policies  must  adjust  to  the 
changing  technology  in  both  the  product  and  the  manu- 
facturing process. 

F.   SEMICONDUCTOR  SUPPLIERS'  AND  CONTRACTORS'  COMMENTS 

In  order  to  obtain  current  views  of  the  JAN  and  other 
government  semiconductor  procurement  policies,  interviews 
were  conducted  with  representative  semiconductor  manufactur- 
ers and  contractors.  Appendix  A  lists  the  questions  asked 
and  about  which  the  interviews  centered.  The  following 
paragraphs  are  a  compilation  of  the  results  of  the  inter- 
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views.  Most  of  the  discussion  centered  on  the  JAN-QPL  pro- 
curement system;  however,  other  policies  naturally  entered 
the  conversation.  There  was  unanimous  agreement  that  the 
JAN-QPL  system  should  be  retained,  however,  changes  are 
needed.  Some  of  the  changes  mentioned  during  the  interviews 
are  contained  in  the  following  paragraphs. 

1.  Procedures  should  be  devised  to  provide  an  incentive 
for  contractors  to  design  in  JAN  products  into  their 
systems.  As  the  system  stands  now,  specifications  can 
be  developed  to  accommodate  only  unique  semiconductor 
devices. 

2.  Qualification  and  testing  procedures  for  new  semicon- 
ductor devices  is  very  expensive  and  time  consuming 
for  suppliers  and  time  consuming.  The  JAN-QPL  system 
does  not  lend  itself  to  qualification  of  technologi- 
cally advanced  semiconductor  devices.  The  qualifica- 
tion process  usually  takes  approximately  two  years. 
By  that  time  commercial  production  of  the  device  is  in 
its  declining  stage  and  the  volume  produced  is  greatly 
reduced.   Figure  1  illustrates  this  point.   DESC  needs 
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Figure  1.   Growth  Phase  of  Typical  Semiconductor  Device 
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to  guarantee  a  specified  turnaround  period  for  product 
qualification  to  help  eliminate  this  situation. 
As  the  program  now  operates,  it  slows  down  the 
introduction  of  new  technology.  When  improvements  of 
a  JAN-QPL  device  are  developed,  the  device  must  be 
requalified  as  if  it  had  never  been  qualified.  This 
requalif ication  consumes  additional  costs,  time  and 
energy. 

Once  JAN  qualified,  there  is  no  guarantee  of  future 
procurements  by  the  government  or  usage  by  contrac- 
tors. Therefore,  with  the  uncertainty  of  purchases, 
suppliers  have  little  or  no  incentive  to  qualify  their 
devices. 

With  the  advancement  of  technology  and  development  of 
more  advanced  integrated  circuits,  current  JAN  testing 
and  inspection  procedures  will  no  longer  apply.  When 
integrated  circuit  assembly  is  totally  automated,  cur- 
rent inspection  procedures  will  actually  be  detrimen- 
tal to  the  reliability  of  the  device. 

Current  rescreeninq  procedures  conducted  by  contrac- 
tors on  incoming  shipments  are  not  cost-effective, 
except  in  rare  instances  (i.e.,  space  systems),  and  in 
the  words  of  one  supplier  are  a  "travesty."  Care  must 
be  taken  that  test  correlation  differences  do  not 
exist  when  devices  are  rescreened.  Feedback  to  the 
supplier  on  rescreening  results  are  a  necessity  for 
future  corrective  action. 

JAN-QPL  semiconductor  devices  do  not  guarantee 
quality.  There  are  no  historical  data  to  compare 
reliability  of  JAN  devices  with  devices  procured 
through  the  other  DOD  approved  systems.  If  a  function 
of  JAN  qualification  is  to  assure  reliability,  as 
implied  in  DOD  Manual  412  0. 3M,  then  other  quality 
assurance  techniques  are  necessary.  Semiconductor 
suppliers  agreed  that  military  specifications  are 
necessary  for  high  technology  devices,  specifications 
that  are  appropriate  for  required  performance 
characteristics. 

More  direct  contact  and  communication  is  necessary 
between  Rome  Air  Development  Center  (RADC) ,  the  custo- 
dian of  MIL-M-33510  and  MIL-STD-383C,  the  contractor 
or  subcontractor  designing  the  component,  and  the 
semiconductor  supplier.  This  dialogue  could  lead  to 
more  utilization  of  standardized  parts  and  state-of- 
the-art  semiconductor  devices.  Semiconductor  specifi- 
cations should  be  created  with  the  advice  of 
semiconductor  suppliers. 
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8.  JAN-QPL  semiconductor  devices  are  substantially  higher 
in  cost  than  commercial  parts  for  the  following 
reasons: 

a.  JAN  devices  are  produced  in  small  uneconomic  lot 
sizes,  resulting  in  many  product  types. 

b.  Specifications  and  inspecting  requirements  are  not 
compatible  with  automation  production. 

c.  JAN  devices  must  be  manufactured  in  the  United 
States  or  a  few  designated  NATO  countries, 
resulting  in  much  higher  labor  costs. 

d.  The  costs  of  maintaining  data  concerning  device 
traceability,  inspection  and  testing  results  are 
added  to  the  cost  of  the  product. 

e.  The  cost  of  capital  investments,  initial  testing 
and  qualification  of  the  product  is  amortized  over 
production  of  the  product.  One  semiconductor 
supplier  estimated  his  company  invested  $40  million 
in  capital  facilities  to  compete  in  JAN-QPLs. 
Another  supplier  stated  the  average  cost  for 
testing  and  qualification  of  a  single  device  is 
$50,000. 

f.  The  required  usage  of  obsolete  and  expensive 
integrated  circuit  packaging  adds  to  the  cost.  One 
supplier  estimated  30%  of  the  military  applications 
of  integrated  circuits  could  utilize  plastic 
packaging  vice  the  more  expensive  ceramic. 

The  estimates  of  price  difference  between  JAN- 
QPL  devices  and  comparable  commercial  products 
varied  from  four  times  the  cost  to  almost  2  5  times 
the  cost  for  a  one-of-a-kind  !S'  (space  level), 
qualified  part.  JAN-QPL  devices  available  from 
only  one  source  are  estimated  to  be  almost  twice 
the  cost  of  competitively  procured  JAN  products. 
One  semiconductor  supplier  commented,  "We  end  up 
selling  the  paperwork  and  giving  away  the  product." 
The  more  data  required,  the  higher  the  cost. 

9.  DESC  needs  to  administer  the  JAN-QPL  program  more 
effectively.  Contractors  have  no  idea  who  is  coming 
up  for  JAN  qualification.  It  was  recommended  DESC 
publish  a  periodic,  updated  notice  of  JAN-QPLs  and 
qualified  manufacturers.  This  notice  should  include 
the  status  of  manufacturers  currently  undergoing 
qualification.  It  was  also  felt  DESC  needs  to  update 
its  data  storage  and  retrieval  capability  and  that  the 
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DOD  needs  to  commit  more  resources  to  the  administra- 
tion of  the  JAN-QPL  program. 

10.  There  was  general  agreement  that  the  JAN-QPL  program 
creates  an  economic  barrier  for  businesses  attempting 
to  compete  for  QPL  contracts.  The  qualification  and 
testing  of  products  is  expensive.  The  required  docu- 
ments on  testing  and  inspection  data  are  enormous  in 
volume.  The  reporting  requirements  to  maintain  quali- 
fication was  cited  as  "overwhelming."  One  semiconduc- 
tor supplier  recently  dropped  35  of  its  135  JAN-QPL 
products,  because  none  were  sold.  Little  incentive 
to  quality  exists  if  there  is  no  demand. 

11.  Utilization  of  a  JAN-QPL  does  not  guarantee  delivery 
of  product  in  a  timely  manner.  Low  volume  QPL  devices 
still  require  long  lead  times  for  delivery  due  to 
infrequent  demands. 

One  contractor,  Lockheed,  has  established  a  "monitored 
line  program"  of  QPL-type  semiconductor  devices.  The  moni- 
tored line  program  (MLP)  was  developed  because  not  all 
specifications  were  covered  by  QPLs.  In  1972  the  Air  Force 
contracted  with  Lockheed  to  establish  and  administer  the  MLP 
to  insure  a  constant  source  of  high  quality,  reliable  semi- 
conductor devices.  Lockheed  supplies  these  parts  to  over  40 
subcontractors  and  other  aerospace  industries.  The 
reliability  rate  and  delivery  record  of  these  parts  is 
unequaled.  To  date  there  have  been  no  failures  of  these 
parts  in  operation  [Ref.  12].  Lockheed  maintains  qualified 
engineers  at  the  suppliers'  manufacturing  sites  to  constant- 
ly monitor  the  processes  and  inspect  the  products.  The  MLP 
seems  to  work  for  their  low  volume,  high  reliability 
requirements.  However,  the  Lockheed  representatives  inter- 
viewed did  state  that  QPLs  are  first  choice  when  they  meet 
all  specification  requirements. 
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This  chapter  provided  an  insight  to  the  complexity  of 
the  semiconductor  procurement  systems  and  the  limitations  of 
the  JAN-QPL  procurement  process  in  this  rapidly  advancing 
technological  sector.  Based  on  research  and  interviews 
conducted,  recommendations  for  improvement  of  the  JAN-QPL 
procurement  process  and  the  QPL  procurement  process  in 
general  will  be  given  in  the  next  chapter. 
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V.   ANALYSES  AND  RECOMMENDATIONS 

A.  INTRODUCTION 

This  chapter  will  provide  a  conceptual  analysis  of  the 
government's  acquisition  process.  Next,  the  QPL  program 
will  be  discussed  within  the  context  of  the  government's 
acquisition  objectives.  An  analysis  of  QPL  procedures  and 
policies  will  be  given  along  with  recommended  applications 
of  QPLs.  Some  variations  of  purchasing  procedures  are 
introduced  for  comparison  purposes.  Specific  recommenda- 
tions directed  toward  the  JAN-QPL  system  for  the  procurement 
of  semiconductors  will  also  be  included,  followed  by  the 
conclusions  of  the  study. 

B.  GOVERNMENT  ACQUISITION  PROCESS 

In  any  government  acquisition  there  always  exist 
tradeoffs  that  must  be  made:  quantity  versus  quality,  high 
technology  versus  reliability/maintainability,  schedule 
versus  test/ evaluation,  etc.  In  making  an  acquisition  the 
government  must  consider  cost  with  regard  to  a  variety  of 
factors,  such  as  delivery  schedule,  reliability,  quality, 
maintainability,  interchangeability  (standardization) , 
performance  (specifications) ,  life  cycle,  expansion  of 
industrial  base,  and  political  realities  (source  selection, 
competition  and  small  business  considerations) .  To  best 
achieve   an   optimum   combination   of   these   factors   at   a 
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favorable  price,  the  procuring  agencies  have  a  multitude  of 
contracting  mechanisms  at  their  disposal.  One  such  contract 
mechanism  is  prequalif ication,  where  higher  procurement  cost 
is  traded  for  expected  higher  quality  and  prompt  delivery. 

The  QPL  was  first  established  for  specific  products 
because:  1)  "the  time  required  for  testing  in  connection 
with  production  would  unduly  delay  delivery  of  the  supplies 
being  purchased."  2)  Either  the  costs  of  repetitive  testing 
would  be  excessive  or  testing  would  require  special 
equipment  not  commonly  available.  3)  The  government 
required  assurance,  prior  toward,  that  the  product  was 
"satisfactory  for  its  intended  use"  [Ref.  6:p.  4].  During 
the  1970' s  the  third  justification  for  a  QPL  was  changed  to 
read,  "it  (the  QPL)  covers  life  survival  or  emergency  life 
saving  equipment."  In  1985  a  fourth  justification  was  added 
which  stated  a  QPL  could  be  established  if,  "the  application 
is  critical;  failure  of  the  part  or  equipment  would  jeopar- 
dize successful  completion  of  the  mission  or  pose  a  signifi- 
cant risk  to  life  or  property"  [Ref.  3:p.  4-4].  Thus  the 
objectives  of  QPLs  went  from  prompt  delivery  and  satisfac- 
tory performance  to  prompt  delivery  and  critical  importance. 
But  what  results  do  QPLs  actually  achieve?  When  should  QPLs 
be  applied?  And  what  changes  to  the  QPL  program  can  be  made 
to  more  effectively  obtain  stated  objectives  and  still  keep 
the  program  cost  effective? 
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C.   QPL  ANALYSIS 

To  begin  the  analysis  of  QPLs  the  following  question 
will  be  discussed:   What  results  do  QPLs  actually  achieve? 

QPLs  provide  an  incentive  for  vendors  to  become  quali- 
fied. In  private  business,  it  is  common  practice  to  narrow 
down  the  supplier  base.  This  is  accomplished  to  provide 
incentive  to  the  vendors  to  become  the  major  supplier  for  a 
consistent  buyer.  QPLs  in  effect  do  this  by  defining  an 
entry  barrier  to  prospective  vendors.  If  a  vendor  goes 
through  the  qualification  process  he  is  saying  in  fact,  that 
he  is  serious  about  manufacturing  the  product.  The  vendor 
is  making  an  investment,  based  on  a  reasonable  assurance 
that  future  acquisitions  will  be  made  and  will  be  profita- 
ble. Once  qualified  the  vendor  has  greater  bargaining  power 
in  seating  price.  Ideally,  from  the  government's  stand- 
point, there  should  be  more  than  one  qualified  vendor.  If 
not,  that  vendor  has  in  essence  a  monopoly  and  the  govern- 
ment will  probably  pay  a  premium  price  for  any  qualified 
product  procurements.  If  there  is  more  than  one  qualified 
source,  prices  may  approximate  competitive  outcomes,  while 
quality  and  service  should  improve. 

QPLs  can  eliminate  undue  delay  in  the  acquisition 
process.  In  certain  cases  where  time  is  the  requirement  in 
specified  conformance  tests  (i.e.,  storage,  aging, 
endurance,  etc.),  QPLs  can  be  justified  in  eliminating 
delays   in   procurement.      When   extensive   or   complex 
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conformance  testing  or  special  equipment,  not  normally- 
available,  is  necessary  for  verification  of  product  confor- 
mance, qualification  testing  can  also  be  justified  to  ensure 
subsequent  timely  product  delivery.  Another  requisite  for 
qualification  involves  the  presence  of  a  high  correlation 
between  conformance  test  outcomes  and  manufacturing 
operating  characteristics,  characteristics  known  and  obser- 
vable only  by  the  manufacturer.  Qualification  is  required 
because  destructive  tests  are  necessary  for  the  government 
to  ensure  the  integrity  of  the  manufacturing  process  and 
thus  conformance  of  the  product  to  specifications.  Without 
qualification  in  these  cases,  normal  post-production  confor- 
mance tests  would  be  unrealistic  in  the  procurement  environ- 
ment. However,  QPLs  do  not  eliminate  delay  when,  after  an 
extended  period  of  time  after  specification  issuance,  there 
are  no  qualified  sources.  QPLs  can  not  be  justified  if  the 
qualification  testing  is  100%  repeated  during  specification 
conformance  testing  during  production.  If  production  lead 
time  is  greater  than  the  time  required  for  qualification 
testing,  qualification  could  be  eliminated  with  all  required 
examination  performed  during  production.  When  qualification 
procedures  have  been  waived  to  secure  "qualified"  vendors, 
the  qualification  requirement  should  be  reviewed  for 
justification. 

QPLs  provide  some  measure  of  quality  assurance.   It  has 
been  brought  out  in  the  QPL  studies  and  it  is  stated  in  the 
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QPL  directives  that,  "the  listing  of  a  product  on  the  QPL 
signifies  only  that,  at  the  time  of  examination  and  test, 
the  manufacturer  could  make  a  product  .  .  .  that  meets  the 
specification  requirements"  [Ref.  3:p.  4-2].  The  1968  DOD 
QPL  report  asked  the  rhetorical  question,  "Does  .  .  .  this 
one  time  manufacture,  sometimes  hand-made,  assure  quality  in 
production?  Obviously  not"  [Ref.  6:p.  8].  However,  justi- 
fication for  qualification  has  shifted  over  the  past  ten 
years  to  include  "life  survival/emergency  life-saving 
equipment"  and  "critical"  parts.  These  justifications 
surely  show  concerns  toward  quality  products.  The  198  0  DOD 
QPL  study  recommended  QPLs  be  justified  only  for  require- 
ments not  critical  in  terms  of  mission  or  safety  [Ref.  3:p. 
4].  The  implication  of  qualification  and  QPLs  on  product 
quality  assurance  is  an  area  of  misinterpretation  and 
controversy. 

Justification  for  prequalif ication  because  an  item  is 
"life-saving"  or  "critical"  to  mission,  should  not  be  an 
item's  sole  justification.  Destructive  testing  of  an  item 
during  the  prequalif ication  testing  can  provide  some 
indication  of  its  quality.  However,  other  quality  assurance 
technologies  should  be  administered  in  addition  to 
prequalif ication  for  these  critical  items.  Appropriate 
measures,  such  as  statistical  testing,  monitoring,  quality 
audits,  source  inspections,  etc.,  help  insure  quality  is 
maintained   throughout   the   production   cycle.     Effective 
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feedback  mechanisms  must  also  be  established  to  resolve 
problems  and  correct  discrepancies.  Supplier  performance 
data  should  be  maintained  including  defective  rates,  quality 
audit  results,  and  delivery  performance,  to  be  used  to 
determine  future  procurement  decisions. 

QPLs  also  have  an  indirect  effect  on  quality  assurance. 
Once  qualified,  a  vendor  desires  to  maintain  his  qualifica- 
tion for  future  procurements.  If  a  vendor  were  subsequently 
taken  off  the  qualification  list,  it  could  have  far-reaching 
ramifications.  This  could  include  unwanted  publicity,  loss 
of  future  contracts  and  resultant  diminished  return  on  his 
investment . 

QPLs  ensure  continuous  availability  of  the  product.  As 
long  as  the  product  is  actively  procured,  the  vendor  will 
manufacture  it.  The  vendor  has  already  made  the  investment 
and  it  would  make  poor  business  sense  not  to  maintain  quali- 
fication. One  semiconductor  supplier  stated  that  his 
company  invested  $40  million  to  manufacture  qualified 
products  for  the  government.  They  are  determined  to 
continue  manufacturing  qualified  products  and  "do  the  job 
right,"  as  long  as  a  market  exists  [Ref.  13]. 

QPLs  can  promote  product  standardization.  QPLs  cannot 
be  expected  to  directly  reduce  proliferation  of  parts  in  the 
DOD  inventory.  But,  QPLs  can  increase  standardization  if 
the  specifications  emphasize  performance  and  interface 
standards  rather  than  component/design  standards. 
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QPLs  can  inhibit  product  innovation.  Whenever  a 
qualified  product's  design,  performance,  or  manufacturing 
process  is  changed,  even  improved,  the  entire  qualification 
process  must  be  repeated.  Considering  all  the  money,  time, 
and  paperwork  necessary  for  qualification,  manufacturers  may 
feel  it  is  not  worth  the  effort  and  continue  manufacturing 
as  previously  qualified. 

QPLs  can  limit  competition  for  direct  government 
procurements.  As  discussed  in  earlier  chapters  QPLs  do 
provide  an  economic  barrier  in  bidding  for  contracts 
involving  qualified  products,  although  recent  changes  to  the 
program  have  alleviated  this  barrier  somewhat  for  small 
business.  Subcontractors  who  have  continuously  provided 
qualified  products  for  contractors,  but  have  not  previously 
qualified,  cannot  be  considered  for  bids  when  spare  products 
on  a  QPL  are  procured  directly  by  the  government.  This 
situation  should  be  corrected;  but  care  must  be  taken  so  as 
to  be  fair  to  suppliers  previously  qualified. 

D.   APPLICATION  OF  QPLS 

As  a  general  guideline  QPLs  will  be  most  effective  in 
the  following  situations: 

1.  acquisitions  for  large  quantities  of  similar  items 
with  relatively  low  unit  cost, 

2.  acquisitions  for  items  in  a  mature  technology  stage, 

3.  acquisitions   for   items   expected   to   be   procured 
repetitively, 
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4.  acquisitions  for  items  that  could  lend  themselves  to 
multiple  applications. 

QPLs  have  been  found  to  be  less  effective  for  the 

following  type  acquisitions: 

1.  Acquisitions  for  complex  items  in  an  immature  tech- 
nology stage.  At  this  technology  stage  qualification 
tests  may  not  be  standard,  potential  qualified  sources 
may  be  extremely  limited,  or  specification  require- 
ments are  changing  too  rapidly.  A  QPL  at  this  point 
would  severely  restrict  innovation  in  product 
improvement . 

2.  Acquisitions  for  low  volume  items.  It  has  been  diffi- 
cult to  obtain  qualified  sources  for  low  volume  pro- 
curements. The  incentive  for  prospective  vendors  is 
not  there  for  an  investment  of  their  resources.  Under 
these  conditions  first  article  testing  and  other 
methods  of  assuring  necessary  demonstration  of 
requirements  would  allow  better  flexibility  in  match- 
ing costs  and  risks  [Ref.  8:p.  15].  If  the  QPL 
process  was  applied  in  this  situation,  the  government 
could  expect  to  pay  a  premium  price  for  the  product. 

E.   ACQUISITION  VARIATIONS  OF  QPLS 

The  following  acquisition  programs  are  discussed  to 
point  out  timely  procurement  techniques  that  attempt  to 
achieve  quality  and  timely  delivery  of  parts. 

1.   Qualified  Manufacturers  Lists 

The  Qualified  Manufacturers  List  (QML)  is  similar  to 
the  QPLs  in  its  objectives.  Its  pilot  test  program  is 
currently  being  conducted  on  the  general  MIL-M-38510 
specification.  Instead  of  discrete  products  being  quali- 
fied, an  entire  line  or  family  of  products  will  be  quali- 
fied. Defense  Electronic  Supply  Center  (DESC) , 
representatives  visit  manufacturing  facilities  to  conduct 
detailed  surveys.    Survey  requirements  include  production 
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inspection  systems,  quality  and  reliability  assurance 
programs,  test  facilities,  production  facilities,  and  line 
certifications.  Verification  that  the  manufacturer  has  an 
effective  self-audit  program  also  is  included.  This  process 
should  be  more  effective  in  several  commodity  areas,  such  as 
the  rapidly  advancing  technology  area  of  semiconductors,  and 
also  in  areas  of  similar  types  of  manufacturing,  such  as  the 
tooling  and  machining  industries.  If  effective,  QMLs  should 
increase  competition  and  reduce  costs.  Product  quality 
needs  to  be  closely  monitored  if  a  large  supplier  base  is 
established.  QMLs  would  be  effective  as  a  buyer's  guide  for 
both  prime  contractors  and  direct  government  procurements. 
A  supplier  rating  system  should  be  used  to  narrow  supplier 
base  and  provide  incentive  for  supplier  performance. 

2 .  Monitored  Product  Line 

Contractors  have  also  developed  parallel  systems 
similar  to  QPLs  and  QMLs  to  satisfy  their  component  and 
parts  requirements.  In  the  previous  chapter,  Lockheed's 
Monitored  Product  Line  was  discussed.  The  effectiveness  of 
these  systems  is  impressive.  However,  contractors  should 
still  be  directed  to  purchase  QPLs  or  from  QMLs  first,  if 
those  products  meet  their  requirements.  Monitored  Product 
Line  should  still  remain  a  supplement  to  QPLs  and/or  QMLs. 

3 .  Chrysler's  Purchasing  Program 

An  example  of  commercial  buying  techniques  is 
Chrysler's  procedures  for  purchasing  integrated  circuits  for 
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their  automobiles.  Chrysler  insures  quality  as  a  standard 
feature  in  their  integrated  circuits  by  the  following 
procedures.   The  company: 

a.  conducts  an  in-depth  yearly  analysis  of  supplier 
quality  performance  and  holds  periodic  meetings  with 
suppliers  during  the  year. 

b.  requires  suppliers  to  qualify  each  component  separate- 
ly. Chrysler  will  not  approve  a  vendor's  complete 
product  line  at  once. 

c.  requires  vendors  to  have  statistical  process  control 
programs . 

d.  develops  long  term  contracts  that  are  not  rebid  unless 
there  is  a  problem.  However,  Chrysler  does  retest  the 
market  yearly  to  ensure  competitive  prices  are 
offered. 

e.  has  initiated  a  vendor  rating  system  to  determine 
future  buying  decisions.  The  concept  is  to  give  the 
best  suppliers  more  business  and  eliminate  poor  per- 
formers. The  rating  system  is  based  on  price, 
delivery,  technology  assistance  and  quality.  Quality 
rating  is  determined  by  defect  rates,  results  of 
quality  audits  of  suppliers'  manufacturing  process  and 
controls,  electrical  performance  of  component  samples, 
and  responsiveness  of  suppliers  to  quality  problems. 

f.  requires  all  integrated  circuit  suppliers  to  license  a 
second  source,  to  ensure  continuous  availability  of 
product.  Current  defect  level  is  .0015  or  1500  parts 
per  million  for  components  received.  [Ref.  14: pp. 
62113-17] 

This  purchasing  program  is  similar  to  QPLs  but 

covers  a  much  more  restricted  variety  of  products.    Some 

concepts  are  found  in  these  techniques  that  are  also  found 

in  the  QPL  program.   These  include  separate  qualification 

for  each  component,  inspection  of  suppliers'  manufacturing 

facilities,  incentive  for  suppliers  to  perform  well,  narrow 

supplier  base,  and  establishment  of  continuous  availability 
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of  parts  through  second  sources.  The  supplier  rating 
concept  that  Chrysler  has  established  to  provide  feedback  to 
suppliers  on  problem  areas,  needs  further  development  for 
application  in  government  acquisitions. 

Having  covered  general  applications  of  QPLs, 
specific  recommendations  will  be  made  directed  at  the  JAN- 
QPL  system  of  semiconductor  procurement. 

F.   JAN-QPL  SYSTEM  RECOMMENDATIONS 

The  following  recommendations  are  based  on  conducted 
research  and  interviews  with  representatives  of  the 
semiconductor  industry,  and  if  properly  implemented  should 
result  in  obtaining  advanced  and  reliable  semiconductors  at 
the  lowest  cost. 

1.  Implementation  of  the  QML  process  in  this  commodity 
area  should  increase  efficiency  and  reduce  the  cost  of 
procurement.  This  system  should  also  provide  incen- 
tive for  product  innovation  and  reduce  the  time  to 
operational  application. 

2.  Implementation  of  the  Military  Drawing  System  combined 
with  the  elimination  of  Source  Control  Drawings  should 
increase  standardization  in  semiconductor  applica- 
tions. With  greater  standardization,  volume  produc- 
tion mighr  be  possible,  thus  permitting  usage  of  the 
best  quality  control  techniques  available  and  reducing 
their  unit  cost. 

3 .  Procedures  should  be  developed  to  provide  incentives 
for  contractors  to  design  JAN  or  Military  Drawings 
components  into  their  systems. 

4.  Specification  conformance  testing  requirements  must  be 
revised  as  technology  advances  and  automated  manufac- 
turing processes  become  commonplace.  As  the  technolo- 
gy continues  to  advance,  rescreening  procedures  and 
quality  assurance  methods  must  also  adapt  to  the 
changing   processes.     Standards   and   specifications 
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should  also  reflect  the  new  technology  and  represent 
realistic  operational  requirements. 

Currently,  military  and  space  systems  incorporate 
semiconductors  that  may  have  operational  lives  of 
thirty  years  or  more.  The  government  has  a  need  to 
procure  semiconductors  as  replacement  parts  for  those 
systems  long  after  the  part  has  ceased  to  be  produced 
for  commercial  usage.  Therefore,  replacement  parts 
costs  are  extremely  high.  Perhaps  preplanned  semicon- 
ductor product  improvement  could  be  used.  It  would 
reduce  technical  risk  during  initial  procurement  and 
allow  for  future  utilization  of  state-of-the-art  tech- 
nology, thus  increasing  effectiveness  and  operational 
life  of  the  system.  It  would  reduce  reliance  on  obso- 
lete technology  and  in  the  long  term  reduce  support 
costs. 

To  help  implement  better  future  procurement  strate- 
gies, the  U.S.  government,  the  U.S.  semiconductor 
industry,  and  U.S.  manufacturers  of  government  end-use 
systems  which  utilize  semiconductors  should  form  a 
board  to  establish  policies  and  objectives  for 
improvement  of  the  current  procurement  system.  [Ref. 
10:p.  vii] 

Because  of  the  increasing  importance  of  semiconductors 
in  major  defense  systems,  the  government  must  provide 
incentives  for  U.S.  manufacturers  to  maintain  onshore 
manufacturing  capability.  All  individuals  interviewed 
agreed,  that  once  U.S.  manufacturers  located  offshore, 
the  U.S.  would  have  no  capability  to  produce  semicon- 
ductors in  a  serious  national  emergency.  The  govern- 
ment should  provide  incentive  to  motivate  the  industry 
to  invest  beyond  efforts  required  to  meet  normal  con- 
contractual  obligations.  This  motivation  could 
include  government  paying  the  costs  for  product  or 
manufacturer  qualification;  or,  as  mentioned  in  a 
recent  Wall  Street  Journal  article,  provide  funds  for 
a  multifirm  laboratory  research  center  [Ref.  15:p.  1] . 
As  the  government  pursues  more  effective  procurement 
procedures,  it  should  strive  to  establish  a  basis  for 
a  partnership.  The  current  procurement  policy  does 
not  address  this  issue. 


G.   CLOSING  RECOMMENDATIONS  AND  CONCLUSIONS 

Many  of  the  recommendations  mentioned  in  past  studies 
still  need  to  be  addressed.    It  is  assumed  that  resource 
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problems  cause  many  of  the  difficulties  and  limitations 
previously  noted  in  the  operation  of  the  system.  However, 
with  the  Congressional,  business  community,  and  public 
attention  the  QPL  program  is  receiving,  it  appears  that 
further  revision  of  the  program  is  in  the  near  future.  The 
QML  program  is  a  good  start.  As  it  is  with  most  programs, 
the  application  of  the  process  will  determine  its  effective- 
ness.  Some  closing  recommendations  include: 

1.  All  QPLs  should  be  reviewed  for  appropriateness.   Many 
can  be  eliminated. 

2 .  Future  use  of  QPLs  should  be  strictly  reviewed  by 
independent  experts. 

3 .  Increased  resources  should  be  devoted  to  the  adminis- 
tration of  the  program. 

4.  Management  data  should  be  developed,   reviewed,   and 
analyzed,  for  future  program  direction  and  policies. 

5.  Periodic  review  requirements  on  the  need  for  existing 
QPLs  should  be  strengthened. 

As  the  program  now  operates,  the  decision  of  vendor  pre- 

qualif ication  should  be  chosen  only  when  short  delivery 

dates  require  it,   or  expensive  time-consuming  tests  are 

involved  in  conformance  testing.   QPLs  are  a  means  to  an  end 

and   tradeoffs   in   costs,   quality   and   schedule   must   be 

expected.   The  decision  to  prequalify  must  be  applied  on  an 

item  by  item  basis,   and  not  solely  because  the  general 

specification  is  prequalif ied.   It  is  not  cost-effective  and 

severely  restricts  competition  and  innovation.   Prequalifi- 

cation  should  be  an  economic  procurement  decision.    When 
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prequalification  is  applied  premium  prices  will  be  paid, 
especially  for  sole  source  products. 

In  the  long  term  prequalification  will  be  less  costly 
when  the  administrative  requirements  are  reduced.  As  a 
means  to  that  objective,  data  repositories  need  to  be 
automated  to  improve  the  acquisition,  storage,  update  and 
retrieval  of  reprocurement  and  technical  data.  Compatible 
computer  data  bases  within  the  government  would  be  required. 
With  such  a  system,  Military  Drawings  and  specifications 
could  be  continuously  updated  and  allow  for  rapid  trans- 
mission throughout  the  U.S.  Widespread  availability  of 
technical  data  and  drawings  would  be  instrumental  in 
increasing  competition,  achieving  greater  procurement 
effectiveness,  and  increasing  standardization  and  cost 
reduction. 
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APPENDIX 
INTERVIEW  QUESTIONS  FOR  CONTRACTORS  AND  SUPPLIERS 


1.  Quotation  from  Defense  Electronics,  February  1986:  "The 
military  uses  yesterday's  technology  today  while  the 
commercial  industry  uses  tomorrow's  technology  today." 
Do  you  agree  with  that  statement? 

2.  Do  you  agree  with  the  statement,  "There  is  a  lack  of 
standardization  in  military  specifications?"  Does  it 
prevent  the  volume  automation  necessary  for  statistic 
quality  control  (SQC)  methods,  thus  resulting  in  higher 
prices  for  the  same  quality?   Why? 

3.  Could  you  explain  the  quotation,  "procurement  procedures 
are  centered  on  testing  for  quality  rather  than  on 
designing  in  quality?"   Do  you  agree? 

4.  What  quality  control  methods  do  you  employ  or  does  the 
government  direct  you  to  employ,  for  QPLs  (i.e.,  100% 
rescreen,  100%  device  testing  at  each  major  step  of 
fabrication)?  Does  it  shorten  product  life?  Does  the 
government  ever  inspect  plant/processes?  Conduct 
quality  audits? 

5.  What  is  the  difference  in  your  cost  between  a  QPL  and 
commercial  product  which  is  for  all  intents  and  purposes 
the  same  specifications? 

6.  Are  any  warranties  offered  with  QPLs? 

7.  There's  a  debate  between  limiting  Mil-Spec  manufacturing 
in  favor  of  standard  commercial  parts.  How  do  you  feel? 
If  commercial  products  were  procured,  do  you  foresee  any 
economic  costs  to  American  industry?  Do  you  see  an 
advantage  in  second  sourcing  electronic  components? 

3.  Do  you  think  QPLs  are  successful  in  reducing  delivery 
delays  as  they  are  designed  to  do? 

9.  What  types  of  contracts  are  used  for  QPLs?  Cost  plus, 
fixed,  etc . ? 

10.  There  is  a  time  lag  from  innovation  to  utilization 
through  QPL  program  in  military  applications,  contrasted 
with  commercial  applications.  How  do  the  time  periods 
compare? 
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11.  Is  being  on  a  QPL  worth  the  effort  (hassles,  paperwork, 
testing  procedures,  inspections,  etc.)?  Is  it  cost 
effective?  What  is  your  motivation  to  be  on  the  list? 

12 .  Who  pays  for  the  QPL  testing?  Do  you  get  a  payback  on 
investment  costs,  time,  effort?  Do  you  have  any  finan- 
cial data  available  (i.e.,  QPL  sales  %  total  sales, 
profit  margin,  ROI,  administrative  overhead  costs, 
dollar  amount  of  annual  sales)?  Any  large  financial 
costs  involved? 

13.  Do  you  know  if  you  are  single  source  or  multiple  source 
for  your  QPLs?  Does  that  affect  price?  Are  you  aware 
of  competition/other  sources  for  QPLs? 

14 .  Do  you  ever  sell  the  same  part  to  private  company  as  to 
government  without  certification? 

15.  How  does  your  company  ensure  quality  in  parts  you 
purchase  or  subcontract  out?  Emulate  QPL  procedures  for 
your  purchases?  If  QPLs  are  purchased,  does  the 
liability  stay  with  the  manufacturer,  even  if  that 
product  is  used  in  your  component?  Do  you  pass  on  the 
QPL  cost  to  the  end  user? 

16.  What  type  of  recertif ication  procedures  are  used  for 
QPLs  that  you  manufacture? 

17.  Do  you  think  QPLs  hinder  small  businesses  from  competing 
for  QPL  government  contracts?   Are  QPLs  entry  barriers? 

18 .  Any  personal  experiences  with  QPLs  that  might  help  this 
research?  Have  the  procedures  changed  recently 
resulting  in  better  effectiveness?  Are  there  any  inter- 
mittent evaluations  conducted  on  products  validating  the 
necessity  of  the  QPL  program?  Should  QPLs  be 
discontinued  after  a  certain  period  of  success  with 
product/process? 
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